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Operational Limitations of Charge 
Transfer Devices 


By K. K. THORNBER 
(Manuscript received November 15, 1972) 


The incomplete transfer of charge and the existence of random noise 
lead to the primary operational limitations of charge transfer devices. 
Owing to the signal dependence of the residual charge, which accumulates 
as a result of the incomplete transfer, signal detection with static detection 
levels becomes seriously tmpaired before the onset of significant signal 
attenuation or noise degradation. A scheme using dynamic detection levels 
as found to greatly extend the operational range of CTD’s and achieves 
the minimum possible error rate for detecting uncorrelated charge packet 
sizes. By contrast, simple coding procedures are found to be ineffective 
an overcoming signal degradation due to incomplete transfer. Shannon’s 
expression for maximum information transmission capacity is trans- 
formed into an expression for maximum information storage capacity. 
It ws found that significantly larger storage capacities are possible with 
CT'D’s than have been achieved. 


PRELIMINARY REMARKS 
Proposition: Devices Function at Their Limits of Operation 


One has only to design or fabricate a device which “‘exceeds specs,” 
and then, because of his success, receive a set of revised (and more 
demanding) specifications from the systems people to appreciate this 
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very basic principle. Or, if one fixed his goals too low, some clever 
fellow will come along with a new twist which makes full use of the 
device and revolutionizes the industry. On the other hand, devices 
function at their limits of operation and no better. This lies behind 
the widely recognized importance of ascertaining fundamental opera- 
tional limitations in the early stages of device development.! 

It is, therefore, the primary objective of this article to discuss the 
operational limitations of charge transfer devices (CTD’s). At a time 
when the basic device structure to be developed is still uncertain, and 
with relatively little analysis of the charge transfer process, noise, error 
rates, storage capacities, etc. (and with even less experimental verifi- 
cation of these analyses), the results of such an article may seem very 
preliminary at best. In some respects this will be the case. However, 
sufficient progress has been made in understanding the operational 
features of CTD’s, especially with respect to incomplete transfer and 
noise, that definite limitations can be placed on how large information 
storage capacity and how small (digital) error rates can be made. 

It is my intention to outline CTD operational limitations in rather 
general (even perhaps philosophical) terms and to refer to appendixes, 
existing articles, work in preparation, etc., for mathematical details. 
The results are optimistic to the extent that they indicate how much 
better one can hope to do with CTD’s than is commonly envisioned. 
However, at the same time, the results are pessimistic in that it is not 
obvious how one is to achieve this optimum use. Implementing our 
dynamic detection scheme, which results in the minimum possible 
error rate for digital signals, represents, nonetheless, a major step 
towards operating CTD’s near full capacity. 


I. INTRODUCTION 


According to a most significant theorem due to Shannon,? the maxi- 
mum information transmission capacity Cy of any channel is deter- 
mined by the bandwidth and the signal/noise ratio of the channel. 
With a slight modification this theorem can be transformed into a 
theorem on the maximum information storage capacity C, of any 
channel, in particular of a CTD. This C, then places an upper limit 
on the number of bits of information which can be stored in an un- 
regenerated section of a CTD. If we restrict consideration to codes in 
which the size of each charge packet is independent of preceding or 
subsequent packets, we can calculate a minimum error rate and 
specify an optimum detection scheme for digital signals. To obtain 
this result, some knowledge of the signal/noise ratio and the accumu- 
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lation of residual charge owing to the incomplete transfer of portions 
of every charge packet 1s necessary. 

In this paper, in order to establish the minimum possible error rate 
and maximum possible storage capacities attainable with CTD’s, it 
will be necessary to first review the way incomplete charge transfer 
determines the attenuation-versus-frequency characteristic and the 
total bandwidth of a CTD. The role of incomplete charge transfer in 
signal degradation will then be discussed. Once the dependence of the 
accumulated residual charge on the preceding signal is understood, 
it is possible to devise a dynamic detection scheme for simple digital 
signals and to compare this with the basic method of absolute-ampli- 
tude or fixed-threshold detection (static levels). With any form of 
detection, noise will introduce errors. Following a review of CTD 
noise, error rates are discussed. It is found that in the presence of 
noise this dynamic detection scheme attains the theoretical minimum 
error rate. Finally, maximum information storage capacities are calcu- 
lated using the bandwidth and signal/noise ratio characteristic of a 
CTD. The methods used to obtain specific results are sufficiently 
general that they can be used, for example, to calculate error rates for 
nonoptimal detection schemes. 


II. REVIEW OF INCOMPLETE CHARGE TRANSFER, SIGNAL ATTENUATION, 
AND BANDWIDTH 


As with other electronic devices, signal attenuation and device 
bandwidth are extremely useful concepts with which to discuss the 
maximum storage capacity and minimum error rates characterizing 
the operational limitations of charge transfer devices. In a CTD, signal 
attenuation arises primarily from the incomplete transfer of charge*~® 
and only secondarily from charge losses, for example, through thermal 
(or other) leakage currents. Let us first consider incomplete transfer, 
then signal attenuation, and finally bandwidth. 


2.1 [Incomplete Transfer 


It is clear that the incomplete transfer of charge from one elemental 
cell to the next will lead to signal degradation.’”:® The character of this 
degradation can be ascertained as follows. The charge Q;,; in the zth 
elemental cell at time ¢ will be the charge in the previous cell during 
the previous transfer cycle (of period r,) diminished by the charge 
Q?_,,-,, left behind in the (¢ — 1)th cell but increased by the charge 
left behind in the 7th cell during the immediately preceding transfer 
Q?,_,,, and also less any charge lost (or gained) during the previous 
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storage, Q_1,_,,. Thus 


Qi. = Oi-1,t—-1, ome og pee ae OQ} re — OLA a (1) 


In general, this equation is very difficult to solve since the Q? and Q! are 
nonlinear functions of Q;. Nonlinearities, however, are common in 
electronic devices. Taking the usual approach, one makes a small- 
signal analysis in order to linearize the equations. Thus we write 
Qit = Qi,o + i,t, Where q:,. is the time-independent (dc) component 
(bias) and q;,: is the small (ac)-signal component. Similarly, Q° and Q! 
can be decomposed into de and ac components. Substituting these 
into eq. (1) we can obtain two equations, one for the de terms and one 
for the ac terms. The de equation can give us the time-independent 
(de) charge bias level at the output. While knowledge of this may be 
important in some applications, it does not lead to any significant 
operational limitations. Of greater importance 1s the solution of the 
equation for the ac terms. 
The equation for the ac terms is from eq. (1) 


Gittttro = Viste — Ge + Gtis — Ge (2) 
As part of the linearization, one takes q3 = paq: and gq} = pBq:. (a and 
6 can be calculated from the coefficients of a Taylor series expansion 
of Q° and Q? in terms of g:pa = dQ*/dQ. and pB = dQ'/dQ., where 
p is the number of individual charge transfers within an elemental cell 
and Q, is the charge to be transferred.) Substituting into (2) one obtains 
the basic equation for qi,t: 


Gittitrro = Git — Page, + Pagdiri,s — DBQi,t (3) 
which becomes upon taking the Fourier transform 
qiti(w)etre = gi(w)(1 — pa — pB) + pagi+i(w). (4) 


Up to this point we have linearized eq. (1) and passed to the fre- 
quency domain, typical procedures in electrical engineering. We pro- 
ceed to solve eq. (4) by first calculating 

: P _1— pa — pp —twTo 
Git (w)/Qi() = [are maentere ° : 
Then we note that since qi:1(w)/q:(w) is independent of 7, it follows 
that qv (w)/qo(w) = [qi+1(w)/gqi(w) ]%, where N is the number of ele- 
mentary cells in the shift register. Recognizing that qn (w)/qo(w) 1s the 
transfer function of the shift register, H(w), one finds that’.® 


H(w) = e-ieNe ( ee (5) 


1 — pae 7 
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As discussed in Section 2.2, H(w) can in principle be used to deter- 
mine gy, for any given sequence of input charge packets q:_n;,, 
Qt—(N+1) to: | 

The first factor in H (w) 1s just the phase delay in the signal present 
even in the limit of perfect transfer (a = 8 = 0). The second factor 
contains a frequency-dependent attenuation and a further phase shift. 
To a good approximation we may write 


H(w) = A(w)eW*#), (6) 
where the attenuation factor A (w) is given by 
A(w) = e-nBe-na—eos wo) (7) 
and where the phase factor ¢(w) 1s given by 
e(w) = Noro t+ nesinwr. (8) 


(n = Np, the total number of charge transfers in the shift register 
from input to output.) With knowledge of the device transfer function 
H(w), we can discuss the attenuation A(w) and then compute the 
device bandwidth. 


2.2 Attenuation 


The attenuation factor in eq. (7) can be interpreted as follows. The 
first factor results from charge loss. If a fraction 6 of charge is lost 
with each charge transfer, after n transfers the fraction remaining is 
just (1 — 8)" exp (—ng) (Gif n6? « 1). Charge loss is clearly fre- 
quency independent. The second factor results from the incomplete 
transfer of charge. For wr, 0, very-low-frequency components, the 
size of adjacent charge packets is approximately the same. Thus the 
charge incompletely transferred at site 7 is nearly compensated by the 
charge incompletely transferred at 7 +1. [—paq; + paqii10 in 
eq. (3).| Thus, apart from charge losses, 9:41 q; and, hence, low- 
frequency components are expected to be attenuated very little. Equa- 
tion (7) bears this out. By contrast, if wr, = 2rf/foma (fX f./2 
where f. = 1/7. is the clock frequency), the attenuation is relatively 
large, exp(—2na). Again referring to eq. (3), f& f./2 implies that 
Qi,, and Qi41,1 are ~180 degrees out of phase and q:,, % — qi+1,t- Thus 
contributions to incomplete transfer add (rather than compensate as 
for low frequencies) and, again ignoring charge loss, eq. (8) predicts 
an attenuation of (1 — 2a)" exp(—2na). Again, eq. (7) bears this 
out. For wr, = 7/2 (f = f./4) the attenuation is exp(— na), an inter- 
mediate case in which the phases of each successive packet differ by 
90 degrees. 
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One further point concerning incomplete transfer should be empha- 
sized. A charge packet which “‘loses’’ a fraction a of its charge in each 
of n transfers might be expected to be attenuated by a factor of 
(1 — a)" } exp(—na). However, eq. (7) for A (w) shows how sensitive 
the actual degradation of a packet is to the presence and nature of the 
other charge packets composing the signal. Thus considering one 
‘Isolated’? charge packet can be very misleading. In Appendixes A 
and B we discuss examples of attenuation in the time domain, and in 


10-2 


ATTENUATION 


10-3 


10-4 
0 f,/4 f,/2 


FREQUENCY IN HERTZ 


Fig. 1—Attenuation versus frequency for CTD’s with 2na of 0.2, 2, 4, 5, 8. 
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Appendix C we consider A(w) in more detail. A(w) is plotted in Fig. 
1 as a function of w for several values of na. (6 = 0.) 


2.3 Bandwidth 


In Fig. 1 we have plotted A(w) for 0 S$ f S f./2, although eq. (7) 
would seem to apply for 0 < f. The reason for this is basic. According 
to Shannon’s sampling theorem,? f,/2, one-half the clock frequency 
(one-half the sampling frequency), is the maximum frequency of the 
signal which can be transmitted. Thus given the clock frequency f,, 
the maximum bandwidth a CTD can have is f,/2. 

Incomplete charge transfer clearly reduces the effective bandwidth 
of a CTD. This is evident from the attenuation plotted in Fig. 1. 
Normally one defines bandwidth by the size of the range of frequencies 
for which the attenuation A exceeds some fraction 6 < 1. A more 
convenient definition from the point of view of information trans- 
mission and storage capacity is that the bandwidth B be given by the 
following expression: 





co PO IE ae Foe 
B= [ Toy at = gett (Bna) , (9) 
where J, is a modified Bessel function.” In Fig. 2, B is plotted as a func- 
tion of na. A slowly varying function, B decreases as (f,/2)-(4arna)—? 
for na>>1. Thus despite the rapid attenuation associated with 
na fy10 (e"y0.5 X 10-4 for f = f./4 and ec” Y0.2 X 10-8 for 
f = f./2), the bandwidth is still approximately 0.09 X (f./2), 9 
percent of its maximum value. The relative insensitivity of 6 to na 


BANDWIDTH / (f, /2) 
=) 
uN 


10-2 10-1 1 101 102 
2n@ 


Fig. 2—Bandwidth versus na for a CTD. 
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has an important effect on the maximum achievable CTD transmission 
and storage capacities as we shall see. 


III. THE ROLE OF INCOMPLETE CHARGE TRANSFER IN SIGNAL 
DEGRADATION 


Even though attenuation and bandwidth are frequency-domain 
(and hence analog) concepts, we shall see in Section VII that it is 
these quantities, along with the signal-power-to-noise-power ratio, 
which are needed in order to calculate the maximum information 
transmission and storage capacity for digital as well as analog signals 
in CTD’s. Nonetheless, it is still helpful to discuss certain features of 
digital signals in the time domain in order to better appreciate certain 
operational limitations of CTD’s. For the next few sections we shall 
ignore charge losses and consider only the much more important effects 
of incomplete charge transfer on the signal. 

Suppose a charge packet of initial size Qi (representing a digital 
‘“‘one’’) follows some sequence of charge packets either of initial size 
@, or of initial size Qo (representing a digital ‘‘zero’’). If a is the coefhi- 
cient of incomplete transfer, then after a single transfer the original 
charge packet has been reduced by a factor of (1 — qa), and, after n 
similar transfers, by a factor of (1 — a)”. In addition, the original 
packet picks up some charge left behind by the preceding packets. 
This residual charge we shall refer to as Qr, which is in general a func- 
tion of the preceding signal. Thus the size of the charge packet repre- 
senting the ‘‘one” at the output is given by 


Q = (1 — a)"Qi + Qr. (10) 


A brief analysis of eq. (10) reveals that the primary operational 
limitation imposed by incomplete transfer is the dependence of the 
residual charge Qe on the preceding signal—the preceding sequence 
of zeroes and ones. Neither the attenuation of the size of the charge 
packet per se nor the accumulation of the incompletely transferred 
charge per se play a primary role. For example, suppose that we are 
using a very simple form of absolute-amplitude detection in which any 
charge packet of size Q > Q = (Qi + Qp)/2 is detected as a “‘one”’ 
and any packet of size Q < Q is detected as a “‘zero.”” Then a one 
preceded by a long string of zeroes will be detected as a zero if na > 0.7. 
Thus, under noiseless conditions we shall have a nonzero error rate in 
cases where the signal attenuation is only one-half (see Appendix A). 
On the other hand, the accumulation of incompletely transferred charge 
can be reduced using zero-net-charge coding. As each bit of signal is 
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coded into the same amount of total charge (distributed between two 
packets), the total amount of incompletely transferred charge from 
each bit is the same. Nonetheless, no significant improvement is ob- 
tained. Indeed, the maximum na for zero errors under noiseless con- 
ditions remains a strong function of the preceding sequence of charge 
packets (see Appendix B). This result is true for analog as well as 
digital signals, as is clear from the discussion in Appendix C. The 
dependence of the residual charge on the previous signal suggests that 
more attention should be given the detection of the signal rather than 
its coding. This we discuss in the next section. 


IV. THE OPTIMUM DETECTION OF SIMPLE DIGITAL SIGNALS 


Let us suppose that some arbitrary sequence of charge packets of 
size Q, for a “‘one’ and Q,y for a ‘‘zero” have preceded the charge 
packet which we now wish to detect. The residue or residual charge 
added to the charge packet of interest can be designated Qe as before, 
where Qz is a function (given in Appendix A) of the preceding signal. 
If the charge packet which we are detecting is in fact a one, then the 
size Q(1) of the packet will be 


Q(1) = (1 — a)"Q1 + Qr. (11) 
If, however, the charge packet is zero, then the packet’s size Q(0) will 
be given by 

Q(0) = (1 — a@)"Qo + Qr. (12) 


One will clearly optimize the detection (even in the presence of noise) 
if one choses for the detection level Qz of the mean of Q(1) and Q(0): 


Qa = (1 — a)"3(Qi + Qo) + Qe. (13) 


If Q > Qa, we say that we have detected a Q: packet, and if Q < Qa 
we say that we have detected a Q» packet. Because Qa is a function of 
Qr which in turn depends on the entire preceding signal, we shall refer 
to this as a dynamic detection procedure. In contrast to Qa given in 
(13), the static detection procedure mentioned in Section III has 
Q, = Q/(1 — a), and Q > Q, implies Q: and Q < Q, implies Qo. 

It is shown in Appendix A that under noiseless conditions this scheme 
of dynamic coding is errorfree regardless of the size of na or of the 
nature of the preceding sequence of zeroes and ones. This again illus- 
trates the role of the dependence of Qz on the preceding signal, which 
we noted at the end of Section III. It is shown in Appendix A that 


Q(1) — Qa= (1a) > SQ, Q0) (14) 
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as is clear from eqs. (11), (12), and (18) as well. The quantity (Qi — Qo) 
may be referred to as the dynamic range of the device. Thus relative 
to the dynamic detection level, Qa, the signal, Q(1) or Q(O), 1s attenu- 
ated as (1 — a)”. [Note that [Q(1) — Qa] and [Qa — Q(0) ] are in- 
dependent of the residual charge Qe. | In the presence of noise, errors 
will clearly be introduced if (1 — a)"(Qi — Qo)/2 approaches the noise 
level. This also shows that, having eliminated the signal-dependent 
residual charge, attenuation now plays an important role in limiting 
device operation. As n increases, this signal attenuation coupled with 
the compounding of noise both reduce the signal-to-noise ratio and 
lead to a reduction in the information transmission and storage 
capacities of the device. However, now it will be for na 4 rather 
than for na } 0.7 that attenuation becomes limiting. 

To set the dynamic detection level Qz, Qe must be realizable. In 
the absence of noise, this is always possible in principle since Qp is 
an explicit function of the known, preceding signal. In the presence 
of noise, Qe determined by eq. (31) also yields, for most cases of 
interest, nearly optimal detection in spite of the possibility that some 
preceding packets may have been incorrectly detected." 

In Section V, we briefly review noise in CTD’s and then in Section 
VI we shall see how this dynamic detection scheme minimizes the 
error rate in the presence of noise. This further stresses the importance 
of detection in optimizing the operation of “simply coded’? CTD’s. 


V. REVIEW OF NOISE 


Noise in charge transfer devices is a fascinating subject which, un- 
fortunately, can be only highlighted in this section.!2-!” Owing to the 
dramatic time dependence of the current during a single charge 
transfer, the noise generated during a single transfer is quite non- 
stationary. Since nearly all theories of noise in solid-state devices 
assume that the noise is stationary,!® it is necessary to redo much of 
the theory taking into account the nonstationary aspect. A time- 
domain analysis has been found to be most convenient, whereas 
standard treatments are carried out in the frequency domain. 

In Fig. 3 the most common sources of noise in CTD’s are categorized. 
At the input, one has full shot noise only if the electrons enter the 
source independently (e.g., if generated by the random arrival of 
phonons in an imaging device or if injected by an emission-limited 
diode). At the output the nonrandom coupling to the clock line is the 
worst source of distortion in some cases. A distinction is made between 
noise generated from transfer processes, typically thermal and trap- 
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Fig. 3—Sources of noise for a CTD. 


ping noise, and that from storage processes, typically leakage and again 
trapping noise. We shall return to this distinction shortly. Since a 
charge packet acquires some noise with each transfer-storage period, 
the noise component increases as the packet is transferred from input 
to output. This we refer to as compounding. Finally, the occurrence of 
clock voltage fluctuations in the presence of fabrication variations in 
the individual capacitances leads to a form of modulation noise. This 
type of noise is also compounded. For simplicity we have left out of 
Fig. 3 a number of less important noise sources. 

Let us now return to the important distinction between storage 
process and transfer process noise.‘ It should be recalled that a CTD 
shift-register performs two functions simultaneously, the transfer of 
charge and the storage of charge. In Fig. 4 we indicate the basic dis- 
tinction between the noise generated from these two processes. In the 
case of storage process (SP) noise, the charge fluctuation generated 
during each transfer cycle in each cell is essentially independent of 
that in any other cell. For transfer process (TP) noise this is not the 
case. Conservation of charge implies that if an excess of AQ 1s trans- 
ferred from one storage region to the next, — AQ is left behind for the 
subsequent charge packet. This introduces a correlation in the noise 
in adjacent charge packets which leads to a suppression at low fre- 
quencies of the spectral density of TP noise. SP noise, by contrast, 
is uncorrelated and, therefore, the spectral density is flat (white). 
This difference between TP and SP noise is important for analog 
applications of CTD’s and is discussed in more detail elsewhere. 

For digital applications we shall need the ratio S/N of the square 
of the signal charge to the mean-square noise charge at the detector. 
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Fig. 4—The distinction between storage process and transfer process noise. 


From the discussion of Section IV [see eq. (13) ] the square of the 
effective signal charge at the detector is (1 — a)?" (Qi — Qo)/2 |?. The 
mean-square noise charge AQ? can be written 


AQ? = AQieult = a)?” n AQ2pH gp(n) “- 2AQtpH pp(n), (15) 


where AQ?, put is the input noise contribution, AQ2p the storage process 
noise acquired by a single packet during a single clock period, AQ?» the 
transfer process noise acquired by a single packet during a single 
charge transfer, (1 — a)?” the attenuation from input to output, 
Hgp(n) the compounding factor’? for storage process noise, and 
Hyp(n) the compounding factor! for transfer process noise. These 
compounding factors are approximately equal to n for na « 1; how- 
ever, for na 2 1 they are suppressed”? by incomplete transfer effects. 
For na > 1, Hgp(n) & (n/ra)* and Hyp FY (2a), both of which are 
much less than n. Essentially the effect of incomplete transfer is to 
attenuate the accumulated noise as well as the signal. Owing to the 
correlation between the transfer-process noise components," Arp 
saturates. For shot noise, AQ?ipu, = GQ, where Q is the (mean) total 
signal charge (Qi — Qo). For thermal noise, AQ2> = %kTC, where T 
is the temperature of the charge carriers and C is the storage capaci- 


tance. For our purposes here we shall ignore other noise contributions. 
Thus we find 


S/N = (1 — a)?" (Qi — Qo)/2 P/AQ?, (16) 
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where AQ? is given by eq. (15). Equation (16) is plotted in Figs. 5 
and 6 for a = 107% and a = 10 for C = 1, 0.1, 0.01, and 0.001 pF, 
for thermal noise only and for thermal and shot noise. We shall use 
eq. (16) in Section VI on error rates and in Section VII on storage 
capacity. 


VI. MINIMUM DIGITAL ERROR RATES 


No one would operate a CTD under conditions where errors in 
detection could occur even under noiseless conditions. However, in the 
presence of noise it is possible for a ‘‘one’’ to acquire sufficient net 
‘negative’ noise charge to be detected as a ‘‘zero”’ even under optimum 
conditions. It is the purpose of this section to calculate the probability 
of making a detection error, and to see to what degree the error rate 
(error probability times clock frequency) is minimized for the simple, 
two-level digital coding scheme by using dynamic detection. 

Suppose that an arbitrary charge packet following an arbitrary 
sequence of charge packets would, under noiseless conditions, be of 
size Q, at the output of the shift register. In the presence of noise the 
probability P(Q) that the observed size of the packet is Q within dQ 
is given by 


P(Q)dQ = explL—(Q — Q.)?/2AQ? ]/ (27 AQ?) *dQ. (17) 


If we are using only zeroes and ones, then the probability P of detect- 
ing a certain ‘‘one’’ as a zero is given by 


a 
Pi= [- P@de, (18) 
where Q, is the detection level (see Fig. 7). In eq. (18), Pi depends 
upon Q, = Q(1) which in turn is a function of the sequence of signal 
charge packets preceding the one [see eqs. (11) and (12) ]. To deter- 
mine the average error probability, P; must now be averaged over all 
possible sequences of signals, in general a very difficult task. 
Let us write eq. (18) in a slightly different form by changing 
variables. 


= Q@a-2) exp (— Q?/2AQ?) 
Pim fo aragyr 
or 
[Qa—Q(1)]/(4.Q2)# 
Py = i e~** 2x / (Qr)3 
or 


Pi = falLQa — Q(1)]/(AQ?)#}, (19) 
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Fig. 5—Signal-to-noise ratio for CTD (a = 107%) with storage capacitance C of 
1 pF, 0.1 pF, 0.01 pF, 0.001 pF. 
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Fig. 6—Signal-to-noise ratio for CTD (@ = 10‘) with storage capacitance C' of 
1 pF, 0.1 pF, 0.01 pF, 0.001 pF. 
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P(Q) 


Qy 


\\ 


Fig 7.—A charge packet of size Q, under noiseless conditions has a probability 
distribution of P(Q) in the presence of noise. The probability that Q < Qg (that an 
error is made in detection) is the area “under’’ P(Q) for Q < Qg. 


where, of course, 


fily) = | " @= 2" 2dr] (Qa)? (20) 
We desire (P;) = (fi), where the brackets ‘‘{ )”’ denote averaging 
Q(1) (and possibly Qz) over all possible sequences of ones and zeroes 
(Q(1) = Q(1 — a)"Q: + Qe from eq. (11) ]. If Q. > Qa for all possible 
Qr (as it must if errors are to be avoided under noiseless conditions), 
then we show in Appendix D that (P) = f{([Qz — Q(1) ]/(AQ?)#)}. 
In other words, the function f evaluated at the average of its argument 
is a lower bound to the average of the function, the average error 
probability. This permits putting a lower bound on the error rate. 
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If [Qa — Q(1) ] is independent of Qp, as is the case for the dynamic 
detection scheme discussed in Section IV, then 


(Pi) = fi{LQa — Q(1)]/(AQ?)?} (21) 
and the average error probability is at the lower bound. For the 
dynamic detection scheme of Section IV, [Q2 — Q(1)] = — (1 — a)” 


X (Q1 — Qo)/2, which implies that 
(S/N)4 
(Pi) = i e~*2dx/(Qr)} , (22) 


where (S/N) is given in eq. (18). If static detection had been used 
[Qa = Q/(1 — a)], then the average error probability would always 
exceed the (P1) given in (22). 

It remains to prove that the optimum (dynamic) detection scheme 
given in Section IV gives the lowest possible error probability for 
simple digital coding. The proof is as follows. We found above that 
for detecting a one the average error probability was at least 


(Pi) = fifL(Qa) — (Q(1))]1/(4Q?)¥}. (23) 


(The ‘‘lb’” stands for lower bound.) Noting the definition of fi [eq. 
(20) ], we note that we can make (Pi), smaller by reducing (Qa), 
(Q(1)) being already determined. However, we must also consider the 
error probability in detecting a ‘‘zero.’’ Proceeding as for a “‘one,”’ 
we obtain for Py the error probability for detecting a certain ‘‘zero,”’ 


Po = fol{LQa — Q(0) ]/(AQ?)3}, (24) 
where now 
fol) = f° em da/(2n)} (25) 
Also 
(Po) = fof{L(Qa) — (Q(0))J/(AQ?)*}. (26) 


From the definition of fo, we note that we can made (P)) smaller by 
increasing (Qa), (Q(0)) being already determined. Assuming that an 
equal number of ‘zeroes’ and ‘‘ones’”’ are used in the simple digital 
coding, then (by symmetry) choosing (Qa) so that (Pi) = (Po)w, 
we shall achieve the minimum lower bounds. (Pi) = (Po) for 
(Qa) = [(Q(1)) + (Q(0)) 1/2. But for our dynamic detection scheme 


(Qa) = Q(1 — a) = [(Q(1)) + (Q(0))1/2 (27) 


and for our dynamic detection scheme (P1) = (Pi) and (Po) = (Po). 
Therefore, since (Qa) for the dynamic detection scheme produces the 
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lowest possible lower bounds for error probabilities, and since the error 
probabilities are in fact equal to these lower bounds, no other detec- 
tion scheme can detect with lower error probability. (It is possible 
that another scheme can do just as well, however, since it is only (Qa) 
and not Qa, itself which is the determining factor.) 

It is clear that this theorem places an operational limitation (a 
minimum error rate in detection) and CTD’s using simple digital two- 
level coding. The theorem can be extended" to the dynamic detection 
of multilevel digital codes. 


VII. MAXIMUM STORAGE CAPACITY 


One use of the CTD is as a memory or storage element. In other 
applications the CTD can be used to shift or to transfer information 
from one location to another. To properly access the operations of 
CTD’s in these applications one must calculate the maximum infor- 
mation transmission capacity and the maximum information storage 
capacity of the CTD. As a result of the work of Shannon, our labors 
are greatly diminished. 

Shannon? proved a most profound theorem. Let B be the bandwidth 
of a transmission channel, and let S/N be the signal-power-to-noise- 
power ratio. Then the maximum transmission capacity of the channel 
Cr in bits per second is given by 


Cr = B loge(1 + S/N). (28) 


This result can be understood for the CTD in the S/N > 1 range 
as follows. The number of levels into which a digital signal can be 
divided and still be detected with reasonably small error is (S/N)}. 
Loge(S/N)? is the maximum amount of information in bits detected 
with each charge packet. f, is the rate at which charge packets are de- 
tected. Thus f, loge(S/N)? = 4f, loge(S/N) ~ B loge(1 + S/N) is the 
number of bits of information transmitted per second. (In Section 2.3 
we noted that for na «1, B® f,/2.) Shannon was, of course, much 
more interested in the S/N < 1 range. For this case his theorem implies 
that no matter how noisy the transmission channel may be, it is 
always possible to pass information along it. We shall not make use of 
Shannon’s result in this latter range. 

A more interesting quantity from the standpoint of the CTD 1s the 
maximum information storage capacity. This can be calculated from 
Shannon’s Theorem? as follows. If C7 is the number of bits per second 
transmitted, then if one waits a time 7’, equal to the time it takes the 
information to be transferred from the input to the output of the 
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linear medium, the maximum storage capacity in bits C’; must be 
given by 
C,=T7Cr = T.B log.(1 + S/N). (29) 


For a transmission line, 7’, is given by the length of the line divided 
by the propagation velocity. For a CTD, T, = N./f. where N, = n/p 
and p is the number of charge transfers per clock period T, = 1/fo. 
Thus for a CTD we find for the maximum information storage capacity 
C’, in bits: 

C, = N.(B/f.) loge(1 + S/N). (30) 


[Strictly speaking, the maximum information storage capacity will 
actually be less than or equal to the C, given in eq. (30). This is be- 
cause as S/N decreases, the length of the code word increases.? How- 
ever, for a CTD with N, storage units, the maximum length of a code 
word is restricted to N,. Thus for small S/N, the prediction of eq. (30) 
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Fig. 8—Storage capacity C, versus n, the number of charge transfers, fora = 1073, 
Q/C = 10 volts, C = 1, 0.1, 0.01, 0.001 pF. Shown for comparison is C, for 2- and 
4-level digital codes. . 


1472 THE BELL SYSTEM TECHNICAL JOURNAL, NOVEMBER 1973 


may be in fact unrealizable. As our primary concern is for S/N > 1, 
the upper limit of C’, should be sufficiently accurate. | 

Knowing N., (B/f.), and (S/N) as functions of n, the number of 
charge transfers, we can calculate C’, versus n to determine the maxi- 
mum C, possible under various circumstances and for what nC, is 
maximum. This has been done in Fig. 8 for a = 107% and in Fig. 9 
for a = 10-4. In both figures Q/C = 10 volts, Q = Qi = 2Q), and 
storage capacitance C' = 1, 0.1, 0.01, 0.001 pF. Also shown is C, 
for two-level and four-level codes. Here n is limited by an n (maximum) 
for each code at the number of transfers beyond which signal degrada- 
tion due to incomplete transfer would lead to errors in absolute- 
amplitude detection in the absence of noise. We note (2) that the 
maximum C, occurs for n about a factor of three larger than for 
n(maximum) from the examples of simple coding and detection, and 
(27) that the maximum value of C, is about a factor of four to five 
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larger than C, at n(maximum) for these common digital codes. [It 
should also be noted that C, at n(maximum) for two-level digital 
coding exceeds C, at n(maximum) for multilevel coding. This is dis- 
cussed in more detail elsewhere.? ] It is encouraging to note that there 
exists such a margin between what is theoretically possible and what 
can be simply accomplished. From Section VI, however, it appears that 
it will be rather difficult to achieve optimum performance if this be 
desired (or essential). 


VIII. CONCLUSIONS AND RECOMMENDATIONS 


It is not surprising to find that incomplete charge transfer and 
random noise (especially shot and thermal) limit the bandwidth, 
storage capacity, and error rate of CTD’s. What is surprising, how- 
ever, is that the residual charge level Qr resulting from the portions 
of charge (preceding the packet of interest) incompletely transferred 
is so strongly signal dependent that signal detection with static detec- 
tion levels becomes serlously impaired prior to the onset of significant 
signal attenuation or noise degradation. Coding to offset the signal 
dependence of Qe is found to be ineffective for the simple examples 
considered. On the other hand, by employing our dynamic detection 
scheme, which adjusts the detection levels to null out the signal 
dependence of the incompletely transferred charge, the operational 
range is significantly extended, limited only by the physically un- 
avoidable effects of attenuation and noise. It is also shown that no 
detection scheme can be devised with a lower error rate than this 
dynamic detection scheme. 

It might be concluded on the basis of the above result that more 
attention should be focused on detection rather than coding as a 
means of offsetting the worst effects of incomplete charge transfer. 
Noting the results shown in Figs. 8 and 9, however, it is apparent that 
substantial increases in storage capacity are possible with more 
sophisticated coding-decoding schemes. 
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APPENDIX A 


In this appendix the degradation of digital signals 1s discussed in 
general in some detail. In Appendix B these results are applied to 
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certain specific cases using several simple coding procedures. In 
particular, the signal-dependent residual charge Qr 1s discussed. These 
mathematical details should be of some assistance in understanding 
several equations presented in Sections III, IV, and VI of the text. 
The analysis will be in the time domain. In Appendix C a frequency 
domain analysis is given. 

We shall denote by Qy the size of the Nth charge packet at the 
input which precedes the packet of interest by N clock cycles. As in 
the text we shall denote by Qe the size of the accumulated residual 
charge originating from the incompletely transferred portions of the 
preceding packets, Qv. Mathematically Qe is given by7:?® 


Ste en wv )axQn (31) 


where n is the total number of transfers from input to output and a 
is the coefficient of incomplete transfer for each transfer. [Kquation 
(31), as well as eq. (3), are somewhat approximate. To obtain a ‘“‘pure”’ 
binomial factor in (31), or equivalently to be able to write a single- 
transfer equation like eq. (3), one must assume that the actual transfer 
of charge can be approximated by simplified single transfers either on 
a per-cell basis as in (31) or on a storage-region basis as in eq. (8). 
The error involved in this approximation will be of the order of @ or 
ne, whichever is larger. | 

The physical significance of (31) is the following. The portion of 
Qn which will show up in Q» are electrons incompletely transferred 
N times, each time introducing a factor a. The binomial factor gives 
the number of distinct alternative sequences of ‘‘transfer’’ or “‘no 
transfer’? which can lead to a portion of Qy contributing to Qr. 

Suppose now, as in the first example in Section III, we have a 


packet of size Q: preceded by an infinite string of packets of size Qo. 
Then for Qr one has 


Qe(000--+) = (1a) & (" Ww )a%Qe 


N=1 


=— (1 —a)"{1 — (A — a)“ ]Qo. (32) 
Similarly, 


Onis) SS a = ee) ends. (33) 


For a Q; following the string of Q)’s, the size of the charge packet 
Q[eq. (10) ] at the output will equal Q = (Qi + Qo)/2 if n is such that 


(1 — a)"Qi — (1 — @)*{1 — (1 — @)~ 700 = (Qi + Qo) /2 
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or if (1 —a)" = 34 [to within a factor of (1—a) #1]. Should 
(1 — a)" > 4, Q < Q and the one would be detected as a zero. 

In Appendix B we shall discuss more complicated coding schemes to 
see whether Q, (31) can be reduced or at least made less sensitive to 
the signal preceding the charge packet of interest. For the present let 
us continue to derive some of the other results stated in the text. 

To east eq. (13) into a simpler form we proceed as follows: 


Qa= (1— a) MES Q, 


ai nf} ee Pee \ 5 _f +6 
= (1a) + (1a) ("1% Jar - 6+) 


~ 9 (1 —a)"— (1 —a)* (1 — Ga )| + Qr 
= Q(1 — a) + Qz, (13’) 


where 


Qe = (1a) © ("1 Jew — Q). (34) 


The static detection level, Q; = Q/(1 — a), actually differs by a 
factor of (1 — a)~! from the Q used in Section III and in the discussion 
following eq. (33). The difference, while insignificant, arises from 
whether one takes Qa to be the average of Qi; and Qo, the sizes of the 
charge packets at the input, or whether one takes Qa to be the size 
at the output of an average charge packet [of size Q@ = (Qi + Qo)/2] 
following a string of similar packets. Thus for such a case 


= lf) = n an n n+wN NQD 
Q=@= G0 -a)r+ (ayn o ("FY Jar 
= Q(1 — a)" = (1 = a)" — Ga) ] 


Q. = Q/(1 — a). (35) 


To show that by using the dynamic level Q,z given by eq. (13) one 
can have zero detection errors in the absence of noise we proceed as 
follows. Using eq. (11) one has at once that 


Q(1) — Qe = (1 — a)"(Qi — Q) (36) 


independent of Qr. As Qi > Qo, Qi > Q and, therefore, Q(1) — Qa > 0. 
Similarly using eq. (12) one finds Qz — Q(0) > 0. Thus, in the absence 
of noise, Q(1) and Q(0) are always separated by Qa, and hence no error 
need be made in distinguishing them. 


or 
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APPENDIX B 


In this appendix we use the results of Appendix A to investigate 
what improvement if any is possible in CTD operation by using several 
simple coding procedures. We shall assume noiseless absolute-ampli- 
tude detection using a static detection level at the average output 
charge level. Equation (31) for Qe can be used to calculate the result 
of other coding procedures. 

In Table I, I have enumerated four simple means of representing 
or coding a digital zero (0) and a digital one (1) using charge packets. 
The first is just to represent a 0 by a Qo packet and a 1 by a Q; packet. 
As calculated in Appendix A [see following (33)], a Q1 following a 
long string of Qo’s [example ‘‘(a)”’ | will be detected as a Qoif na > 0.7. 
This is the ‘‘na Limit” entry in the table. Finally, the size of the Q, 
packet is attenuated as exp(—na) as stated. For this coding a second 
example, “‘(b),” is given—a --- Q1Q0QiQ0 --- sequence. In this case 
Qr is always sufficiently large for a Q; and sufficiently small for a Qo 
that under noiseless conditions Q(1) > Q and Q > Q(0) for any na. 
However, as noted in Section 2.2, such a signal is attenuated as 
exp(—na) attenuation. 

One might hope that by preventing Qrz from becoming much differ- 


TABLE IJ—Four SIMPLE MEANS OF REPRESENTING DIGITAL 
ZEROES AND OnEs Using CHARGE PACKETS 


Example Representation na Limit* Attenuation 

0 1 
Qo Q1 

(a) 1000 --- 0.7 e7na 

(b) 1010 --- 00 e72na 
QoQo Q:01 

(a) 1000 -:- (1) 0.7; (2) 1.67 ene 

(b) 1010 --- (1) 0.79; (2) 2.4 i 
QQ. = QoQ 

(a) 1000 --- (1) ~; (2)0.8 etna, gona 

(b) 1010 --- (1) 2.386; (2) 0.785 e7ne 
QQ% QQ 

(a) 1000 --- (1) 0.4; (2) 0.5 e7ne 

(b) 1010 --- (1) 1.6; (2) 3.1 gn 


* The notation (1) refers to the first of the two packets forming a bit, and (2) refers 
to the second. 
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ent than Q, one could increase the na limit. In Table I three possi- 
bilities are given. The first consists merely of coding 0 into two ad- 
jacent Qo packets and 1 into two adjacent Q,’s. If one detects the 
second of the two Q: packets in sequence (a), the na limit is increased 
to 1.67. For sequence (b) detecting the second Q; now has an na limit 
of 2.4 while the signal is attenuated as exp(—na). How much of an 
improvement this offers, however, is questionable. To store the same 
amount of information » must be doubled reducing the 1.67 to an 
effective 0.835. To maintain the same information rate, f., the clock 
frequency must be doubled. This will increase a: if a is doubled, then 
the 1.67 limit, already reduced to 0.835, will be reduced further to 
about 0.42. Compared with the 0.7 limit of the simplest code, this is 
rather unfavorable. One compensation is that having two packets to 
detect rather than just one can be used to reduce the error rate in- 
duced by noise. However, one can do better, as the following example 
illustrates. 

The third example in Table I is the zero-net-charge code. Here a 0 
is coded as a Qo packet followed (in time) by a Q packet. (In the 
register shifting from charge left to right this is represented as Q1Qo.) 
A 1 is coded as Q.Q;. The advantage of this procedure is that each 
pair, whether coding a 0 or a 1, contains the same amount of charge, 
2Q. This prevents a buildup of charge in Qe. The most demanding 
test 1s sequence (b) in which the na limit is 2.36. This is a significant 
improvement over the 1.67 limit in the previous example. However, 
if one takes into account that to contain the same amount of infor- 
mation 7 must be doubled (as now each bit requires two charge packets) 
and that the clock frequency must be doubled to maintain the same 
data rate (which will increase a), one realizes that really very little 
has been achieved by increasing the upper limit on na from na < 0.7 
to na < 2.4 (2.4/4 = 0.6). Other straightforward modifications of the 
basic 0, 1 code, of course, suffer from the same fault. Thus to achieve 
any improvement it is necessary that one still must be able to take 
advantage of the possibility of detecting both charge packets to do 
better than the simplest code. The reason for the failure of the zero- 
net-charge code in terms of frequency-domain concepts is given in 
Appendix C. 

One final example is to follow the Qo or the Q: with an intermediate 
packet of size Q. As seen in Table I, sequence (a) puts an na-limit of 
0.5, which is inferior to the other codes. This attempt to reduce 
[Qs — Q| by following Q> or Q; with a Q packet to “average” out the 
incompletely transferred charge is thus seen to be ineffective. 
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APPENDIX C 


It is quite informative to briefly discuss in the frequency domain 
the effects of various digital coding schemes on the character of the 
signal.” 

In Section 2.1 we noted that incomplete charge transfer leads to a 
frequency-dependent attenuation A(w) given by 


A(w) = exp[—nea(1 — cos wr.) | (37) 


for 8 = 0 in eq. (7). A(w) 1s plotted in Fig. 1 for various values of na. 
As discussed in Section 2.2, low-frequency components (f < f./2) 
suffer very little attenuation, whereas components with frequency near 
half the clock frequency (f  f,/2) are attenuated by exp(—2na), a 
large attenuation for na 2 3. 

One can offset this high-frequency attenuation by the following 
scheme. If one takes every other charge packet and replaces it by a Q; 
if it originally was a Qo, and by a Qo if it originally was a Q,, then 
relative to Q one essentially multiplies each packet in turn by +1, —1, 
+1, —1, +1, —1, ---. This has the effect of converting the spectrum 
of the signal from F(f) to F(f./2 — f): the f = 0 component is 
attenuated as A(f,/2) and the f = f,/2 component as A(0). To better 
preserve the entire signal, one can sum the outputs of a register with 
attenuation A(f) and a register with attenuation A(f,/2 — f). The 
ratio of maximum attenuation to minimum attenuation is thus 1m- 
proved from exp(—2na) to 2 exp(—na)/[1 + exp(—2na) |. However, 
distortion near f = f,/4 is still significant for na > 2. 

To see the effect of the zero-net-charge coding scheme on the signal, 
consider this example. If the clock frequency is f,, the maximum fre- 
quency the CTD can carry is f,/2. However, if two charge packets 
are devoted to each 0 or 1 as in the second through fourth examples 
in Table I, then the bandwidth is reduced to f,/4. If the second ex- 
ample is chosen, then the band extends from f = 0 to f = f,/4; if the 
third example (zero-net-charge coding) is chosen, then the band ex- 
tends from f = f,/4 to f,/2, the lower-frequency components of the 
signal being carried at the higher frequencies and vice versa. If 
amplified by exp(-++na), the ultimate effect of incomplete transfer on a 
signal coded using zero-net-charge coding is seen to be essentially the 
same as that on a signal coded using the second example. What is most 
striking, however, is that by reducing the clock frequency by a factor 
of two and using simple coding, one reduces na by a factor of four, 
greatly reducing the attenuation. 
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By examining the effect of other coding schemes on the spectrum 
of the signal, and by taking into account the frequency-dependent 
attenuation accompanying charge transfer in CTD’s it is possible to 
ascertain whether an improvement in (noiseless) detection will be in 
fact real or only apparent. 


APPENDIX D 


In noise, detection, and communication theory one often encounters 
an integral of the form 


I(A) = in e~*Pda/(2r)} , (38) 


where A > 0. This expression, while extensively tabulated numeri- 
cally, is difficult to work with analytically. In this appendix we shall 
(t) bound (A) between two simple analytic functions of A which 
differ by only a factor of 2, and (iz) prove that ([(A)) 2 I((A)) for 
A= 0. 


(t) Bounds on [(A): 


In Fig. 10 we illustrate the motivation for our approximations. 
I(A) is the area under the Gaussian for x = — © tox = — A. If 
we draw a line tangent to the Gaussian at « = — A and extend the 
line from 2 = — A to the z-axis as shown, the area of the triangle 
formed by this tangent, the z-axis, and the vertical line x = — A is 
clearly less than [(A). Similarly, if an exponential curve [B exp(+ C2) ] 
also tangent to the Gaussian at x = — A and decaying to the left is 
drawn, then the area between this curve and the z-axis forx S$ — A 
is clearly greater than [(A). Thus, if we calculate these two areas, we 
will have an upper and lower bound on /(A). (These curves will 
clearly not cross the Gaussian if A = 1, the inflection point of the 
Gaussian. ) 

To calculate the areas we proceed as follows. The slope of exp(—x?/2) 


at zr = — Ais A exp(—A?/2), and of course its value at x = — Ais 
exp(— A?/2). Thus the equation of the tangent is 
y(x) = exp(—A?/2) + A exp(—A?/2) (a + A) (39) 


(which is zero for x = — A — 1/A, f(—A — 1/A) = 0) and of the 
exponential 1s 


y(z) = expL—A?/2 + A(z + A)]. (40) 
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Fig. 10—Approximating the area under a portion of Gaussian curve by bounding 
the area between that of a right triangle (whose hypotenuse is tangent to the Gaussian 


atx = Ta and by the area under an exponential (also tangent to the Gaussian at 
z= —A). 


Thus computing the appropriate areas we find that if A = 1 then 


D/2 < (27r)#I(A) < D, (41) 
where 


D = exp(—A2/2)/A. (42) 


Such bounds are very useful in calculating error rates, where one is 
seldom interested in accuracy better than a factor of two, and where 
upper and lower bounds are often very useful. 


() ([(A)) 2 I((A)), A 2 


To evaluate (J(A)) is clearly very difficult even under the simplest 
of probability distributions of A, whereas [((A)) is generally very easy 
to compute if (A) is known. J({A)) can then be used as a lower bound 
for the more interesting (I(A)). We shall now prove the above in- 
equality. (This result is reasonably well known.” The proof is given 
here for completeness.) 

I(A) is a function of A. According to the mean value theorem we 
may write 
(A) = 104) +2] (a -— (4) +5 

{A) 
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where A’(A) lies between A and (A) and depends on A. Thus we may 
write 





(A) = 1A) +3 ral, A AY 
Now then 
So = A exp(—A?/2)/(27r)} (45) 


which is zero or larger for A 2 0. Thus if we are averaging A over a 
probability distribution P(A) for which P(A <0) =0, then 
A’(A) 2 0, and, consequently, the second term on the right-hand 
side will be zero or greater. Hence it follows that 


(I(A)) 2 I({A)). (46) 


In Section VI this inequality is used to put a lower bound on the error 
rate for detecting digital signals. 
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A Geometric Theory of Intersymbol 
Interference 


Part I: Zero-Forcing and Decision-Feedback 
Equalization 


By D. G. MESSERSCHMITT 
(Manuscript received May 14, 1973) 


A linear-space geometric theory of intersymbol interference 1s intro- 
duced in this paper. An equivalence between the structure of intersymbol 
enterference and a wide-sense stationary discrete random process 1s 
demonstrated and exploited to demonstrate the equivalence of zero-forcing 
(decision-feedback) equalization to minimum mean-square error linear 
interpolation (prediction) of a random process. This equivalence 1s used 
to quickly derive the properties of these equalizers and give them additional 
geometric interpretation. Results from prediction theory are used to 
develop practical computational methods of determining the tap-gains of 
the infinite equalizers for both rational and nonrational channel power 
spectra. Finally, the theory of reproducing kernel Hilbert spaces is used 
to develop a theory of equalization for nonstationary channels with non- 
stationary noise. | 


I. INTRODUCTION 


The analysis of digital communication systems from a geometrical 
viewpoint—the viewing of waveforms as points in a signal space and 
the identification of cross-correlation with the formation of an inner 
product—is by now well established. To a large extent, this approach 
has been popularized by the book of Wozencraft and Jacobs.1 However, 
when it comes to analyzing systems with intersymbol interference, 
frequency-domain techniques have almost exclusively been relied upon. 
The purpose of this paper is to consider pulse-amplitude modulation 
(PAM) systems with intersymbol interference from a geometric 
standpoint, and more specifically to develop a geometric theory of 
equalization. 
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Consideration of the geometric structure of intersymbol inter- 
ference leads immediately to the observation of a striking correspond- 
ence to the theory of minimum mean-square error (MMSE) linear 
estimation of a wide-sense stationary discrete-parameter random 
process. The fact that the latter subject is almost exclusively treated 
by geometric methods? is further impetus for this approach to 
equalization. 

The theories of linear zero-forcing equalization and decision-feedback 
equalization are well established. The properties of linear equalization 


r(t) 7, : THRESHOLD . 
~ AS i * 


\ h(t- kT) / 


t=kT 
ne = ee 
ALENT— ae 
TO 







~B, d(t - kT) 





Fig. 1—(a) Communication system model. (b) Matched-filter receiver. (c) Zero- 
forcing equalizer. (d) Decision-feedback equalizer. 
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are summarized by Lucky, et al.,* while the present state of knowledge 
of decision-feedback equalization is summarized by Monsen® and 
Price.* The primary analysis tools which have been used are the cal- 
culus of variations in the case of linear equalization and Toeplitz forms 
in the case of decision-feedback equalization. 

In this paper, the geometric approach enables us to treat the two 
types of equalization simultaneously using the same mathematical 
framework, in which the relationship between them becomes very clear 
and many of their known properties are given an additional geometric 
interpretation. Many of the results follow directly from the theory of 
MMSE estimation. In addition to the unification and reinterpretation 
of previously known results, the geometric approach leads to exten- 
sions of the theory in several directions. Among these are the deriva- 
tion of an orthogonal expansion in Section 2.4 which is useful in many 
problems involving intersymbol interference, the development of 
practical iterative techniques for determining equalizer tap-gains 
(the infinite case) in Section 3.4, the extension of the theory of equali- 
zation to nonstationary noise and a time-varying channel in Section 
IV, and numerous results on the minimum distance problem associated 
with the performance analysis of the Viterbi algorithm maximum 
likelihood detector in a companion paper.’ 

This paper together with a companion one’ expand upon an earlier 
talk. Readers desiring a limited and short treatment of this subject 
may wish to refer there. The geometrical approach to intersymbol 
interference was also employed to a limited extent in the author’s 
thesis.° 


1.1 Problem Statement 


We will consider the detection of a sequence of digital data digits, 
B,, each assuming one of a finite and predetermined number of levels, 
from the reception 


r(t) = = By,h(t — kT) + n@) (1) 


as determined from the communication system model of Fig. la. It 
will be assumed initially that n(é) is white Gaussian noise (this assump- 
tion will be relaxed in Section IV).' A simple matched-filter receiver 
for the reception of r(¢) is shown in Fig. 1b. In the first of two equiv- 
alent formulations of this receiver, the reception is cross-correlated 


t The assumption of Gaussian noise is not necessary for the majority of results to 
follow, and in particular those which involve only second-order statistics of the noise. 
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with h(t — kT) and the decision on B, made by applying a series of 
thresholds to the result; in the second formulation the cross-correlator 
is realized as a filter with impulse response h(—t) (commonly called a 
matched filter) whose output is sampled at t = kT. The matched- 
filter receiver is optimum when there is no intersymbol interference, 
but in the presence of intersymbol interference the matched filter will 
respond to more than a single data digit and the performance of the 
receiver will be degraded. 

When there is intersymbol interference, a common approach is to 
build a linear filter, called a zero-forcing equalizer (ZFE), which re- 
sponds to only a single time-translate of h(t) (this can only be approxi- 
mated in practice). The most common form of this equalizer, shown 
in Fig. lc, is a matched filter followed by transversal filter (MFTF). 
As N —o the tap-gains of the transversal filter can be chosen such 
that the threshold input is a function of only a single data digit. It is 
important to note for future reference that the MFTF can also be 
modeled in the manner of Fig. 1b as a cross-correlation of r(¢) with a 
linear sum of time translates of h(t), 


N 
>, Omh(t — mT). 
N 


n=— 


The decision-feedback equalizer (DFE) embodies a slightly different 
philosophy in which the DFE forward filter is allowed to respond to 
past (but not future) translates at A(t); the residual interference from 
past data digits is then subtracted out prior to the decision threshold 
using past decisions. A realization of the DFE using again the MF TF 
approach is shown in Fig. 1d. The tap coefficients are now chosen to 
null the response to future data digits; this can be accomplished as 
N— o., 

The shortcoming of both the ZFE and DFE is that their linear 
filters remove intersymbol interference without regard to the effect 
on the noise; the result is that in eliminating the intersymbol inter- 
ference (or a portion thereof) they necessarily enhance the noise.! It 
seems clear intuitively that since the DFE eliminates interference 
from only future data digits, it has more degrees of freedom than the 
ZFE and should therefore be capable of less noise enhancement. A 
proof that this 1s always the case has been given by Price ;* his method 
was to determine an explicit formula for the DFE S/N ratio using 


TIn addition, the DFE is susceptible to decision errors. The effect of errors will 
not receive consideration here. 
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Toeplitz form theory and compare it with the known S/N ratio of the 
Z¥E.* Additional interpretation of this result will be given in Section 
3.1. 

A review of some requisite material on linear spaces and MMSE 
linear estimation is given in Sections 2.1 and 2.2. Readers familiar 
with this material are nevertheless urged to scan these sections for 
notation to be employed in the remainder of the paper. The ZFE and 
DFE are reformulated in Section 2.38. In Section 2.4 the relationship 
between intersymbol interference and MMSE estimation is discussed, 
and a useful orthogonal expansion arising out of this relationship is 
derived in Section 2.5. 

Section III develops a geometric theory of the ZFE and DFE. 
Conditions necessary and sufficient for the existence of these equalizers 
are given in Section 3.1, their performance is discussed in Section 3.2, 
a useful property of the DFE with regard to its output noise sequence 
is interpreted in Section 3.3, methods of calculating the tap-gains are 
derived in Section 3.4, and the relationship between finite and infinite 
transversal filter equalizers receives consideration in Section 3.5. 

Sections II and III are concerned with additive white noise ex- 
clusively. Section IV extends the theory to colored Gaussian noise, 
nonstationary Gaussian noise, and a time-varying channel using the 
theory of reproducing kernel Hilbert spaces (RKHS). 


II. AN EQUIVALENCE TO DISCRETE RANDOM PROCESSES 


The structure of the intersymbol interference in (1) will now be 
shown to have an equivalence to a wide-sense stationary random 
process. The starting point will be a quick review of linear spaces and 
of linear mean-square error (MMSE) estimation of a random process. 


2.1 Hilbert Space Notation 


An inner product space £ consists of a linear space together with a 
defined inner product (x, y) between two elements x and y. All spaces 
in this paper are Hilbert spaces, which consist of an inner product 
space satisfying an additional closure property (specifically, the limits 
of Cauchy sequences must be in the space). The inner product induces 
a norm, or “length” of a vector, 


Iz] = (2) (2) 


and the notion of the distance between two vectors, ||z — y||. The 
geometrical interpretation of these quantities is illustrated in Fig. 2. 
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le <x,y> wl 


IX Ih 





Fig. 2—Interpretation of inner product, norm, and distance. 


A subspace of £ is any set of vectors which itself constitutes a linear 
space. If x,, k € I is a countable or finite sequence of vectors, then 
we denote by M(z2;, k € J) the closure of the subspace consisting of 
all finite linear combinations of elements of the set {z,,k € I} and 
call this the subspace spanned by the w,’s. It is convenient to think 
of elements of M(x, k € I) as convergent (possibly) infinite sums 
of the form 

> EVE 


kel 


even though in some obscure cases not all elements can be expressed 
in this way. 

In many minimization problems it is desired to find the element 
of some closed subspace M which is closest to a vector y; the resulting 
element is called the projection of y on M, is denoted by P(y; 4), 
and satisfies the orthogonality property 


(y — Py; M),x) = 0 (3) 


for all z € M.* The geometric interpretation of (3) is shown in Fig. 3 
for a one-dimensional subspace spanned by 2; for this case the pro- 
jection must be a scalar times x and the validity of (3) 1s apparent. 


2.2 Review of Linear Mean-Square Interpolation and Prediction?’ 


We will now quickly review the theory of linear mean-square 
estimation of a random variable. 

The set of random variables with zero mean and finite variance is a 
linear space, since the sum of any two such random variables itself 
has these properties. This set is also a Hilbert space with inner product 


t When, as in (3), a vector is orthogonal to every vector in M, it is said to be 
orthogonal to M. 
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x 


Ne : <y,x> _X_ 
P(y,M (x}) = lI x || II x [I 


Fig. 3—Projection on subspace spanned by z. 


where £(-) denotes expected value. It is standard to suppress the 
sample space dependence of a random variable as has been done in (4) 
because the geometric properties (inner product and norm) are deter- 
mined by the value of the random variable on the whole sample space; 
that is, by its statistics in their entirety. 

Consider now the following interpolation problem: Suppose that a 
Sequence of zero-mean random variables X,, —~ <k < ©, with 
finite variances are given and it is desired to estimate X based on the 
observation of X;, k ~ 0. If the estimate is further stipulated to be 
linear, it is the same as requiring that it be an element of M(X;,, k ¥ 0). 
Suppose that the estimate Xo is to be chosen in such a way that the 
mean-square error between Xo and the estimate is minimized: 


e min E(Xo — Xo)? (5) 
X0EM (Xz,k #0) 


From (4) and the previous section, the MMSE linear interpolator is 
Xo = P[Xo, M(Xi, k ¥ 0)), (6) 


the projection of X) on M (Xz, k ¥ 0). 

A second estimation problem which will be of interest is the pre- 
diction of Xo based only on X;, k > 0 (an anticausal prediction). The 
MMSE linear predictor is the projection of X» on the subspace 
spanned by X;, k = 1, 2, ---, denoted by P[ Xo, M(X:, k > 0) ]. 


2.3 Zero-Forcing and Decision-Feedback Equalization 


We are now prepared to restate the problem of determining the ZFE 
and DFE filters in a linear space context. It will be assumed that the 
basic pulse A(z) in (1) has finite energy (i.e., is square integrable), 


” W2(t)dt < 0. (7) 


~~ 


The set of waveforms which satisfies (7) is a linear space, which we 
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denote by Le. Le is also a Hilbert space with inner product 


yy) =f e@y oat (8) 


for any two L2 waveforms x(t) and y(t). For the same reason that the 
sample space dependence of a random variable was suppressed in (4), 
the time dependence of the waveforms x(t) and y(¢) has been sup- 
pressed on the left side of (8): it is the entire time waveform which 
determines the geometric properties. 

The class of filters’ which will be considered will be limited to those 
which can be modeled as an inner product (or cross-correlation) of the 
reception r(t) with some Ly waveform. A ZFE is a filter corresponding 
to a waveform g;(t) which does not respond to any translate of h(t) 
except A(t — kT), 


i h(t — mT)g,(t)dt = 0, mk, (9) 
but does respond to h(t — kT), 
J ” A(t — kT) gp (tdt ¥ 0, (10) 


in order that there be a signal on which to base the decision. It is 
evident that if go(t) satisfies (9) and (10) for & = 0, then they are also 
satisfied by gz(t) = go(t — kT) for k # 0. Written in inner product 
notation, (9) and (10) become 


(hi, Jo) = 0, k # 0, (11) 
(ho, Jo) * 0, (12) 


where we have written h;, for h(t — kT). The analogous condition for a 
DFE forward filter is 


(hi, go’ = 0, k> 0, (13) 
(ho, go) ~ 0. (14) 


The forms of the ZFE and DFE in this symbolic notation are shown 
in Figs. 4a and b. The output of the linear filter is a function of By 
(a single data digit) fora ZFE and By_m, m > 0 (all past data digits) 
for a DFE. The tap-gains of the feedback transversal filter storing 
past decisions for the DFE are equal to the responses of go to previous 
pulses, (go, hm), m > 1. 


T In the case of the DFE, we refer only to the forward filter. 
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Fig. 4—Symbolic representations of the two equalizers: (a) zero-forcing equalizer ; 
(b) decision-feedback equalizer. 


2.4 A Congruence Relationship 


Two Hilbert spaces which display an identical geometrical structure 
are said to be congruent!" or unitarily equivalent.’ Specifically, in 
order for two Hilbert spaces to be congruent, there must exist between 
them a one-to-one and onto linear mapping which preserves norms 
and inner products. Although the elements of two such spaces may be 
quite different entities, when considered as elements of their respective 
Hilbert spaces they have the same geometrical structure. 

Define the autocorrelation function of the pulse sequence, 


Ri, = (hin; Nintk): (15) 


It follows from the inequality 


N 2 N WN 
0 = > Om! tem — De » OL mO n LL kn —kn 
m=0 m=0n=0 














that {&,} is a nonnegative definite function. Therefore, there exists 
a second-order discrete random process {X;,} which has autocorrela- 
tion Ri, 
(Xn, Xmpk) = HE (Xm, Xm+e) 
= Ry. (16) 
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For the random process defined in (16), M(hi, k € I) and M(X,,k € I) 
are congruent through the obvious mapping 


N N 
6 | & antag | = 2 amX rn (17) 


which is a unitary linear transformation. To verify this, observe that 
the mapping is linear, preserves norms, 


N 
|6( alten) 


2 N 


2 
>; AmX km 


m= 


N WN 
ba > mn Le km—ken 
m=ln=1 




















2 


(18) 


N 
x mM km 


m=1 














and preserves inner products by an equally simple derivation. 

The mapping of (17) is only defined for finite sums. When I is an 
infinite set, @ can be extended to all of M(hz, k € I) by taking limits 
in the mean. For any f © M(h;z, k € I) there exists a sequence { f;}, 
each consisting of a finite sum of the form of (17), such that f, — f. 
Since $(fx) is a Cauchy sequence from (18) ,we define ¢(f) as the 
limit of ¢(f,), which is in M(X;, k € I) by completeness. 

There is an additional congruence which is useful. From the defini- 
tion of R, in (15), we see that 








Pee A ; | H (ce) |2e%2*P dey 
1 a f{T ; 
== a ip Pldeset? (19) 
where 
oe) 2 
Ro) * > H(« ae m= ) 
=T YX Reinet, (20) 


where R(w) is an equivalent power spectrum of the channel. From (16), 
R(w)/T is the power spectrum of the random process {X;}. Let 
I2(—17/T, «/T; R) denote the Hilbert space of all complex-valued 
Lebesque measurable functions f(w) with domain |w| < 1/7 which 
satisfy 


FOP = 5. [| IF) PR@)do < » (21) 
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with the obvious definition of the inner product. A frequently invoked 
congruence is between M(X;,, -~ <k < ©) and Lo(—727/T, r/T; R).2 
By implication, L2(—7/T, 7/T; R) and M(h,, —7 <k< ©) are 
also congruent through the mapping 


v o cnlinn = 5 ageiobn? (22) 


m=1 m=) 


as is readily verified. 

In the remainder of this paper, the congruence demonstrated in this 
section will be exploited to demonstrate that many available results 
on MMSE interpolation and prediction theory are directly applicable 
to the equalization problems posed in Section 2.3. 


2.5 An Orthogonal Expansion 


The congruence relation of Section 2.4 will be used in this section to 
establish an orthogonal expansion in M(h,, — © <k < ©) which 
will be particularly useful in the sequel. 

Define the element 


et & hy — P[hi, M(An, m > k)] (23) 


which is the difference between a translate of h(t), hz, and its pro- 
jection on the subspace of translates to its right. It will be shown later 
that this element is of particular significance to the DFE. For the 
moment, however, note that e¢ is equivalent to the MMSE prediction 
error of X; based on Xm, m > k, since the projection is the optimum 
linear predictor. It is well known’ that the successive prediction errors 
of a random process are uncorrelated random variables. The equiv- 
alent statement relating to e# is that 


(Civ, On) = [leo ||?Sm,n (24) 


and it is an orthogonal sequence.t This is readily demonstrated directly 
by noting that e* is orthogonal to M(h:, k = m), which contains e- 
forn > m. Hence, (24) follows for n > mand by symmetry for n < m 
also. 

From (24) it follows that as long as 


lle" || > 0 (25) 
the sequence 
Wn = ef/llefll, - 2 <n<», (26) 


is an orthonormal set in Le. The significance of (25) is that the equiv- 


* The norm of ef is independent of k since ef is a time translate of ef. 
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alent random process must not be linearly predictable with vanishing 
mean-square error (in the language of Ref. 3, p. 564, X, must be 
“regular,” or ‘‘nondeterministic’’). 

Expanding h,, in a Fourier series in Wn, 


lhe = Un + Vn 


Un a > CmWant+m 
Cm £ (CT ree hn) = (Win ho) 
(Un) Wm) = 0, -—- eo <n< m7, —-a I m< oa, (27) 


where v,, is the remainder. Equation (27) can be simplified by observing 
that 
(Wm, ho) = 0, m < 0, 


since ho © M(hy, k 2 0) and wm is orthogonal to M(hi, k 2 m+ 1), 
which contains M(h;, k 2 0) whenm < 0. In addition, it can be shown 
(Ref. 3, pp. 571-575) that v, = 0, since the spectrum under consider- 
ation here is absolutely continuous.' Thus, (27) reduces to 


hn = x CmWntm 
m=0 
Cm = (ho, Wm). (28) 


The expansion of (28), which is used in the theory of linear prediction,?3 
is similar in spirit to a straightforward Gram-Schmidt orthogonaliza- 
tion process, but is much more useful in that the coefficients of the 
expansion are independent of n. The main shortcoming of the expan- 
sion (28) is requirement (25). 

The formula for cm given in (27) is not very useful in explicitly 
evaluating the coefficients of (28). A more useful method of evaluation 
is to observe that it is a spectral factorization problem. Defining the 
bilateral z-transform?! of the autocorrelation, 


R*(2) = YL RnB", (29) 
we claim that 
R*(z) = S CnZ” 2 ja. (30) 
n=0 n=0 


' This is by virtue of the fact that integral (21) is in terms of R(w)dw; ie., the 
underlying measure is presumed to be absolutely continuous with respect to Lebesque 
measure. 

+ Note that we define the z-transform in positive powers of z. 
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where the c, are given by (28). To show (30), first calculate R; from 
(15), 


R; = (ho, hj) 
= EE entalWmy Wien) 
Sentai G20 (31) 


0 


nr 


8 


2 CnOnt+5; j3 <0. 


n=—j 
Similarly, the right side of (30) can be manipulated, 


co 


d CrCmL"™ = > = CrCntma”™ + > CrCrame ™, (32) 


m=0 m=0 n=0 m=l1 r=—-m 


Mes 


| 
o 


nr 


and comparing (31) and (82), (80) is established. The representation 
of (30) is not unique. However, Doob (Ref. 3, p. 160) shows that 
the coefficients of (27) uniquely satisfy (30) when the additional 
conditions 


>, CnrZ” € 0, (Z| <1, (33) 
n=0 


Ms 


c< a (34) 


l 
oS 


n 


are required.' The necessity of (84) is obvious from (27), while the 
reason why (338) is needed is that otherwise (380) could be satisfied 
on the unit circle by another sequence with a larger zeroth term, 
contradicting the fact that 


= led. (35) 


Equation (85) follows from the observation that M(h,, k 2 n) = 
M(w,,k 2 n) and therefore P[h,, M(hi,k 2 n+ 1)] = Se CmWarm 
or 


er = CoWn. (36) 


A simple example will serve to illustrate (30). Suppose A(t) has an 
exponential autocorrelation with 


Ry, = Ala, 0<A <1. (37) 


t Of course, condition (25) is also required. 
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Direct calculation of (29) reveals that 


1 — A? 


* a 
E"(@) = G—AnG- AD (38) 
which is in the form of (80) with 
Cr, = V1 — A? A*. (39) 


The validity of (89) can be demonstrated directly for this simple 
example by noting that 


ex = hy, =F Ahrsi (40) 


(as can be verified by showing that e; is orthogonal to hm, m 2 k + 1) 
and thus 


vp. a tim = Alms 
[am — ARm4+al| 
hi Alien 
oa on" 
From (28), 
Cm = (ho, Wn) 
= V1 — AA" (42) 


agreeing with (39). 
The procedure for higher-order rational spectra is equally simple. 


From (29) and the fact that R, is real and even (R_m = R»), it 
follows that 


R¥*(z) = Rt (=): (43) 


Thus, for every zero a; and pole b; of R*(z), a7' and b;7' are also a 
zero and a pole respectively. Thus, R*(z) can be written in the form 


R*(z) = K‘*2! . (44) 

Il (1 — biz) (1 -#) 

ii | z 
la;|, |b, | <l 
so that from (30) 

00 | 3 (1 = 0:2) 
C(z) = ¥ enZ* = VK =*>—____ ., (45) 

Ca (1 = 6,2) 

~=1 


where (33) has been insured by the choice of zeros in (485). 
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When &*(z) is not rational, a more general method of determining 
the coefficients of (28) is required. For this purpose, we use the equiv- 
alent power spectrum of (20). The first form in (20) is the one re- 
quired for analytically determining C'(z), whereas the second form is 
the one which would usually be used in numerical calculations. The 
relationship of R(w) to R*(z) is, of course, 


R(w) = TR*(e?), (46) 


the evaluation of R*(z) on the unit circle. The equivalent of (30) for 
R(w) is 
Rw) _ 


00 ; 2 


k =0 


(47) 








Intuitively, (47) requires the expansion of VR(w)/T, with an arbitrary 
phase characteristic, In a complex Fourier series with only positive 
frequencies. Following Doob (Ref. 3, p. 161), expand log VR(w)/T 
in a Fourier series, 


I fi (w) = wh T 
5 log 7 = 2, re? (48) 





This is always possible because, as will be demonstrated later, in order 
for (25) to be satisfied, it is necessary and sufficient that log R(w) be 
integrable. Define 





g(a) =r+2 >» rye" (49) 
and note that 
Re g(e%#7) = 5 log ite). (50) 
We claim that 
C(z) = e9) (51) 
satisfies (47), since 
[C(eH#?)| = exp[Reg (e#7)] = ./ 2. 


Equation (83) is also satisfied since g(z) is analytic for |z| < 1. 
Equation (51) is an analytic solution to the problem initially posed, 
but a practical means of applying it numerically is required. It 1s 
shown in Appendix A that the Fourier coefficients of (48) can be 
calculated efficiently and accurately using the fast Fourier transform 
(FFT) algorithm. The second difficulty is in determining C'(z) from 


1498 THE BELL SYSTEM TECHNICAL JOURNAL, NOVEMBER 1973 


g(z) in (51). This is easily resolved by noting that 
1 d™ 








S > 
{. -¢@” 
= —— —_ > 
ee qn 9 (2) _ m2 1 (52) 
Co = e7? 


and applying Leibniz’s differentiation rule 


d” me n\ dr-™y, d™ 
dan Y= 2 (") da™—™ dam 


to the product 


d” qr dg (z) 
ness =— ___. 2) es 
dz” “@ dz! (ce dz ) 


= ‘ — ‘soe ee 


mM dg™-™ dz™ 





and, setting z = 0, 


Cn = 


pa Ww) 


n—l 
> (n— mM)ra-mn, 2 21. (53) 
m =0 


Equations (52)—(53) give us a practical recursive method of determining 
the coefficients of (28) when the channel spectrum is not rational. 


III. GEOMETRIC THEORY OF THE ZERO-FORCING AND DECISION-FEEDBACK 
EQUALIZERS 


The zero-forcing equalizer (ZFE) and decision-feedback equalizer 
(DFE) have been introduced in Sections 1.1 and 2.3. In this section, 
we will describe fully the characteristics of these equalizers in the 
context of the geometric structure developed in Section II. 


3.1 Conditions for the Existence of the ZFE and DFE 


The existence of a ZFE and DFE will now be related to the inter- 
polation and prediction of the equivalent random process defined in 
Section 2.2. This relationship will then be used to obtain directly the 
known conditions for their existence. 

The first observation is that the subspaces M(hi, k #0) and 
M(X;,, k # 0) are identical, as are the subspaces M(hi, k > 0) and 
M (Xx, k > 0). The element 


€o9 = ho — Pl ho, M (hi, k a 0) ] (54) 
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is the same as the interpolation error vector defined in Section 2.2, 
(Xo — Xo), while the prediction error vector is the same as 


eg = ho — Plo, M(hu, k > 0)]. (55) 


These two vectors are likely candidates for a ZFE and a DFE because 
they are orthogonal to the subspaces M (hi, k ¥ 0) and M(Ax, k > 0) 
respectively [see Section 2.1 and eq. (8) ]. Hence, they satisfy (11) 
and (13) respectively. To verify that they are indeed a ZFE and a DFE, 
conditions (12) and (14) must be checked. Noting that é) is orthogonal 
to M(hz, k # 0), we have 


(€0, ho) = (€0, ho — PLho, M (hz, k ¥ 0) J) 


= |leo||? (56) 
by definition (54). Similarly, it follows that 
(e"; ho) = |lec|I?. (57) 


Thus, we see that a necessary and sufficient condition for ey (e°) to be a 
ZFE (DFE) is that |leo|] > 0 (|led|| > 0). By definition, the projection 
of ho on a subspace is the element of that subspace which is at a mini- 
mum distance from ho, and hence |leo|| and |les|| are the minimum 
distances between hy and M(h,;, k # 0) and M(h;, k > 0) respectively. 
Since |leo|| can only vanish if hy C M(hy, k # 0), and similarly 
for lle ||, it follows that eo (ef) is a ZFE (DFE) if and only if 
ho E M(hz, k ~ 0) [ho E M(hi, k > 0)]. Physically, these conditions 
mean that h(t) must not be representable as an infinite weighted sum 
of a subset of its own translates. Geometrically, it is evident in Fig. 5 
that, as long as |leéo|| > 0 (or |let|| > 0), eo (or eo) will have a com- 
ponent in the direction of ho and the equalizer will have a response 
to the desired signal. 


ey OR egt 





P(ho,M (hy, k#O))) > 
OR e 
P(hy .M (hy, k > 0)) 


Fig. 5—Geometric interpretation of the zero-forcing equalizer and decision-feed- 
back equalizer. 





M (hy, k #0) 


1500 THE BELL SYSTEM TECHNICAL JOURNAL, NOVEMBER 1973 


The weighting functions (54)—(55) can, under reasonable conditions, ! 
be written in the form of a convergent linear sum of translates of ho, 


Co = ho _ u, Axhr (58) 
e¢ = ho — x at hi (59) 


for some coefficients af. This demonstrates that these two elements 
are just the matched filter followed by transversal filter (MFTF) 
discussed in Section 1.1. It will be shown in the next section that the 
MFTF has particular significance, in that it maximizes the S/N ratio. 

In general, there will be many ZFE’s and DFE’s other than (58)—(59). 
An example of a different ZFE is the element 


ho — P[ho, M(hi, k 2 0)] 
for any hy such that 
(ho, ho) ¥ 0 
ho E M(hx, k ¥ 0). 


An interesting question that arises is, then, whether there ever exists 
a ZFE and DFE when their corresponding MFTF’s do not exist. To 
see that the answer is no for the ZFE (the proof for the DFE is 
identical), note that if ho © M(hz, k ¥ 0), then any go orthogonal to 
M(hi, k #0) is also necessarily orthogonal to hot Thus, we have 
proven the following theorem: 


Theorem 1: The following five statements are equivalent: 


1. ho € M(hi, k ¥ 0) [ho E M(Ay, & > 0) ]. 

2. |leol| > 0 [llecl| > 01. 

3. There exists a ZFE [DFE |. 

4. There exists a ZFE [DFE ] of the form of eg. (54) Leg. (85) ], the 
MFTF. 

5. The random process defined in (16) cannot be linearly interpolated 
[ predicted | with vanishing mean-square error. 


The fifth condition of Theorem | follows from our earlier identification 
of ey and ef as the interpolation and prediction errors, respectively, 
of the equivalent random process. This observation also enables us to 
pull from the literature formulas for the norms of é) and eg. The follow- 


tT This will be discussed fully in Section 3.5. 
We also make use of the trivial observation that any go satisfying (11) is orthog- 
onal to Af (hi, k ¥ 0). 
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ing corollary follows directly from the known formulas for the inter- 
polation and prediction errors of a random process,? 


lel? = | 5° [| R*w)do |” (60) 
lel? = pexp| 5- [7 log R(w)do | (61) 


Corollary 1: A ZFE [DFE | exists if and only if R-(w) [log R(w) ] ts 
integrable. 


Both conditions relate to the fashion in which R(w) vanishes. In 
particular, both require that R(w) vanish on at most a set of measure 
zero. The relationship of (60) and (61) will be discussed more fully 
in the sequel. 

It should be noted also that (61) follows directly from the orthog- 
onal expansion of Section 2.5. From (35) we know that, |legt||? equals 
cg, while (52) gives a relation for cp. When the Fourier series of (48) 
is inverted and 7p is substituted into (52), (61) results. 


3.2 Performance of the Equalizers 


It will now be shown that the MFTF among all ZFE’s and DFE’s 
maximizes the §8/N ratio and minimizes the error probability in white 
Gaussian noise. The derivation will be a simple application of the 
Schwarz inequality. 

Assume that the additive noise in (1) is white and Gaussian. Then 
the decision axis which is applied to a threshold is, for the ZFE, 


(go, 7) = Bo(go, hod + (go, 2), (62) 


where (go, ) = No is a Gaussian random variable with mean zero and 
variance 


N 
Eng = > Ilgoll? (63) 


and N,/2 is the two-sided spectral density of the noise. The minimum 
probability of error decision strategy is then to apply (go, 7) to a series 
of 14 — 1 thresholds, with the specific thresholds depending on the 
probability law on B,. For any such law and series of thresholds the 
probability of error will be a monotone decreasing function of the 
S/N ratio, which is proportional to 


(go; ho)? 2 
S/N « 64 
(N © “Tool oD 
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since (go, 2) 1s a zero-mean Gaussian random variable with variance 
proportional to ||go||?. Noting from (11) that go is orthogonal to 
P{ho, M(hy, k # 0) ] whenever gy is a ZFE, (64) can be rewritten 


S/N « Wu < Iie (65) 
gol? 

by the Schwarz inequality, with equality if and only if go equals eo 
(the MFTF) within a multiplicative constant. Thus, the MFTF, 
among all ZFE’s, maximizes the 8/N ratio. By the same method an 
identical result can be demonstrated for the DFE, if it is assumed 
that the decision-feedback mechanism correctly cancels the tails of 
earlier pulses. 

The preceding derivation, which is a generalization of the Schwarz 
inequality derivation of the matched filter, has the geometric interpre- 
tation of Fig. 6. In writing (65), the maximization of (64) is restricted to 
those go which lie in the hyperplane orthogonal to P[ho, M (hx, k ¥ 0) ]. 
Since every ZFE is also orthogonal to this vector, it follows that the 
hyperplane so described contains the set of all ZFE’s. However, the 
maximization over elements of the hyperplane does not guarantee a 
result which is a ZFE. The vector in the hyperplane which has the 
greatest component in the direction of ho per unit length is evidently 
the one which lines up with éo, as verified by (65). Fortunately, this 
vector also turns out to be a ZFE, so that the maximization is complete. 

An additional observation relative to (65) is that the maximum 
S/N ratio is proportional to ||eo||? for the ZFE and |le"||? for the DFE. 
The maximum §$/N ratio is therefore directly proportional to the mean- 
square interpolation and prediction errors of the equivalent random 
process. Thus, the maximum §8/N ratios of the ZFE and DFE are 
given by (60) and (61) respectively, while the factor by which the 









M (h,, k #0) 


\ 
\ 


\ 
PLANE ORTHOGONAL 
TOP (hy, M(h, .k #0) 


Fig. 6—S/N ratio maximized by the MFTF. 
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——- DISTANCE 
TO x- y PLANE 


DISTANCE TO x- AXIS 
Fig. 7—Geometric interpretation of eq. (66). 


S/N ratio is reduced relative to an isolated pulse with matched filter 
detection 1s obtained by dividing by Ro, the isolated pulse energy. 

Price® derived (61) by a different method and used the geometric 
mean inequality for integrals to show from (60) and (61) that 


lleoll? S lec |l?. (66) 


This important result implies that (z) the S/N ratio of the DFE MF TF 
always exceeds that of the ZFE MFTF,? and (77) a DFE exists when- 
ever a ZFE exists [the contrary is not true, as demonstrated by the 
important example of algebraic zeros in R(w)® |. Using the geometric 
method we have developed, two interpretations of (66) can be given. 
First, it is intuitively apparent that the mean-square interpolation 
error of a random process will be smaller than the mean-square pre- 
diction error, because an interpolation is based on more information; 
similarly, there will be some processes for which interpolation, but not 
prediction, with zero mean-square error is possible. Second, since 
M (hi, k ¥ 0) contains M(hx, k > 0), the distance between hy and 
M (hi, k ¥ 0) (equal to |leo|/?) must be smaller than the distance be- 
tween ho and M(hz, k > 0) (equal to |les||?). This second interpreta- 
tion is a rigorous way of establishing (66) by a method more direct 
than the integral inequality. It has the geometric interpretation of 
Fig. 7, where the distance between a vector fy and the larger subspace 
(the x-y plane) is less than between ho and the subspace it contains 
(the «x axis). 

The performance of the ZFE and DFE can be evaluated for any 
particular channel spectrum using (60)—(61). In particular, (60)—(61) 
can be evaluated in closed form for rational spectra. A different ap- 
proach, which allows us to evaluate the tap-gains of the equalizers 
as well, will be pursued in Section 3.4. 


' This result neglects the effect of decision errors on the DFE. 
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3.3 On the DFE White Output Noise Property 


As observed by Price,® the DFE forward filter is identical to the 
“whitened matched filter’? employed by Forney” as the first element 
of his maximum likelihood detector. The property of this filter which 
is essential to Forney’s application is that the noise sequence at the 
filter output 1s uncorrelated. As with the other properties of this filter, 
this one has a simple explanation in terms of the relationship to linear 
prediction. 

Identifying ef as ef (t — kT), the noise sequence at the DFE for- 
ward filter output is (e*, ). Since n(t) is white noise, this sequence 
will be uncorrelated if and only if 


(et, ex) = 0, mn. (67) 


The validity of (67) and an interpretation of this result in terms of 
the uncorrelated nature of the successive prediction errors of a random 
process has already been given in Section 2.5. 


3.4 Determination of Tap-Gains 


In this section, we will use the orthogonal expansion of Section 2.5 
to derive methods of determining the tap-gains of the forward and 
feedback filters of the MFTF DIE. For comparison purposes the 
well-known relation for the tap-gains of the ZFE will also be briefly 
developed. 

If we write the weighting response of the MFTF ZFE as 


Aoglo = a arhr, (68) 


where the tap-gains of the transversal filter are a,, —- ~ <k< o~, 
condition (11)—(12) becomes 


(€0, hm) = [leo|?bm,o 


~ Yin: (69) 
Taking the bilateral z-transform of (69), 

Go|leo||? = A (z)R*(z), (70) 
where A (z) is the z-transform of the tap-gains 


A(z) 2 2 aye". (71) 
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Thus, from (70), 





_ Qol[70]]? 
A(z) = R*(z) : (72) 
This filter is illustrated in Fig. 8a. When h(¢) is applied to the input 
of a matched filter and the output sampled at a rate of 1/7, the output 
has z-transform R*(z). The transversal filter weighting response has a 
z-transform proportional to R*(z)—, so that the output is consistent 
with (69). 

The S/N ratio of the ZFE, given by (60), is readily derived from 
(72). Writing the relation for tap-gain zero, 


_ 1 FAG, _ aoleol? (de 
OO oa og Daa PREG): oe 
and solving for |leo||?, we immediately get (60) using (46). 


As an example, for the exponential autocorrelation of (37), (72) 
becomes 


A(@) = aoledl? | — ~Aaet + 745-7] 














from which we get 


1 — A? 
2 = 
lleol| 1+ A? 
AoA (75) 
ee 


a, = 0, | k| > l, 

a result derived by Tufts!® by another method. This example points 
out that it is not ever necessary to actually evaluate (60) when the 
channel spectrum is rational, but rather the performance can be 
obtained by equating the zero-order tap-gains of (72) in the manner 
of (73). 

The situation with the DFE is only slightly more complicated. In 
this case the DFE filter is 


Ag egt = > arhk, (76) 


where only taps on one side are involved. Substituting from (28) and 
(36), 


ioe] 
Ag CoW at > CmWarm 
0 m=0 


k= 


Wm > Ait Cm—ky (77) 
k=0 


m=0 
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kT 
R* agllegil2 
h(t) wie O 0 (z) oo agllegll?5k 9 


MATCHED TRANSVERSAL 
FILTER (a) FILTER 
kT 
h(-t) 
h(t) H*(w) O O 





U———_ 


MATCHED 
FILTER 






FORWARD TRANSVERSAL 
FILTER Co(Clz)-Co) 


FEEDBACK 
(b) FILTER 


Fig. 8—Spectral representations of the MFTF zero-forcing equalizer (a) and 
decision-feedback equalizer (b) 
and equating coefficients, 
Agr Co, m=0 
m 
DD bem = (78) 
0, m > 0. 
From (78) we get a recursion relation for the tap coefficients which is 
useful for nonrational spectra, 
m—l1 


Hees Fares, (79) 
Co k=0 


and a z-transform relation which is useful for rational spectra, 


+ 
ae _ Ao Co 
At@) = Oe, (80) 
where A+(z) is the z-transform of the tap-gains of (76). Performing 
(80) again for the autocorrelation of (37), 


At(z) = ag (1 — Az) 
lec I =o = 1 — 4 


which is consistent with (40) and is larger than |leo||? by a factor of 
(1 + A?). As with the ZFE, the performance of the DFE can be 
determined for rational spectra without the explicit evaluation of (61). 

The comparison of (80) with (72) is interesting, in that they are 
identical except for the fact that in (80) C(z) is substituted for R*(z) 





(81) 
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in (72). The annulus of convergence of A(z) will always include the 
unit circle, since R*(z) converges in an annulus containing the unit 
circle. Similarly, C(z) is analytic and nonzero in a region containing 
the unit disk, and hence At(z) will have only positive powers of z 
and converge in a region containing the unit disk. Note that these 
properties of At(z) are critically dependent on (33) being satisfied. 

The spectral factorization method of determining the tap-gains of 
the DFE was given by Monsen} for rational spectra. Price® gave a 
formula valid for arbitrary spectra, but it is difficult to evaluate 
numerically. Since (79) is valid for arbitrary spectra, the method 
presented here represents a synthesis of the appeal and computational 
simplicity of the spectra factorization method with the generality of 
Price’s Toeplitz form result. 

We also need the tap-gains of the feedback filter for the DFE. 
From Fig. 4, the required feedback tap-gains are given by eg’, h_n), 
1<n< o. From (386) and (28), 


00 
b, = (es, h_») = Co Ds Cm{Wo, Wines) 
m=0 


= CoC€n. (82) 
Thus, the frequency response of the feedback filter is given by 


= bm2™ = cof. C(z) — co]. (83) 


The z-transform representation of the DFE just derived is illustrated 
in Fig. 8b. When an isolated pulse h(t) is applied to the matched filter, 
the sampled output has z-transform R*(z). The transversal filter 
multiplies by A+(1/z) = co/C(1/z), as can be verified from (76).' The 
z-transform of the forward transversal filter output is coC(z) because 
of (30), which verifies the causal response which is characteristic of 
the DFE. The output of the feedback filter of (83) is then subtracted, 
to yield (hopefully) a delta function response c?. The reader can verify 
that when the threshold is replaced by a gain of 1/ci (the noise-free 
case) the response is as represented. 


3.5 Pinite Transversal Filter Equalizers 
The previous sections have considered the rather idealized case of 
infinite transversal filter equalizers. Since only finite equalizers can 


T This is because (76) is not in the form of a convolution sum. This distinction was 
not relevant to the ZFE due to the symmetry of that filter. 
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actually be implemented, the important question arises as to when 
and in what sense the infinite equalizer can be approximated by a 
finite one. 

We have already seen in the example of the exponential autocor- 
relation that the infinite equalizer can degenerate into a finite trans- 
versal filter for some channel spectra. This will happen whenever A (z) 
and A+(z) are finite polynomials in z. From (72) and (80) we see that 
this will occur whenever R*(z) is a rational function which has no 
zeros (only poles). When the spectrum is not rational, or is rational 
with zeros, it will be necessary to approximate the infinite MFTF. 

It is straightforward to generalize the results of Sections 3.1 and 
3.2 to subspaces spanned by a finite number of translates of ho. In 
particular, if we replace the criteria of (11) and (13) by 


(hz, Jo) = O —-NzkK&EN, k #0 (84) 
for the ZFE and 
(he, Jo) = 0 lSskSN (35) 


for the DFE, we are left with the consideration of the finite dimen- 
sional subspaces M(hi, -N Sk SN,k #0) andM(hi,1 Sk SN), 
which we will write as My and M# respectively. Then the MFTF 
equalizers which satisfy (84) and (85) are similar to (54) and (55), 


eo(N) = ho — P(ho, My) (86) 
eg (N) 4 ho — P (ho, Mj). (87) 


It is straightforward to see that Theorem 1 can be replaced by the 
following version: 


Theorem 2: The following four statements are equivalent: 
1. ho € Mw [ho E M¥]. 
2. lleo(N)|| > 0 [lle (N)|| > 01: 
3. There exists a ZFE [DFE] in the restricted sense of (84) [(88) ]. 
4. There exists an MFTF ZFE [DFE | tn this restricted sense. 


The question of when it can be asserted that |leo(N)|| > 0 and 
let (N)|| > 0 deserves consideration. The condition that hy © M¥ re- 
quires that coefficients {am, 1S m S N} exist which satisfy 


N 
ho = D0 Omhm. (88) 


This occurrence will be precluded if the set {hn, —- © <m< @} is 
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linearly independent. Similarly, linear independence is sufficient for 
a ZFE to exist in the sense of (84). The following lemma, which is 
proven in Appendix B, establishes sufficient conditions for the linear 
independence of {hmn, - © <m< a}; 


Lemma 1: The following two conditions are sufficient for the linear in- 
dependence of {hm —- 7° <m< ~}: 


1. |leo|| > 0 or |leg|| > 0. 
2. There exists an interval [a,b ],a < b, such that R(w) > 0, € [a, b ]. 


The first condition of Lemma I satisfies our intuition that if an infinite 
MFTF ZFE or DFE exists then the finite MFTF version should also 
exist. The second condition assures us that the finite equalizers also 
exist under much weaker conditions. 

The following theorem establishes a relationship between the finite 
and infinite equalizers, and is proven in Appendix B: 


Theorem 8: As N—~, |leo(N)||? is monotonically decreasing and 
approaches ||eo||?, and likewise for ef (N). Furthermore, ||eo(N) — eo||? > 0 
and |e" (N) — eq"||? > 0. 


The primary conclusion of Theorem 3 is that the infinite equalizer 
can be approximated with arbitrary accuracy (in the sense of Lz 
convergence) by a finite equalizer. In addition, it asserts that the 
S/N ratio of this finite equalizer is greater than that of the infinite 
equalizer ; however, this desirable property may be entirely or partially 
offset by any residual intersymbol interference. 

Each member of the sequence of equalizers guaranteed by Theorem 
3 has different tap-gains, because the projection on a different sub- 
space is being taken with each N. A more aesthetically pleasing ap- 
proximation results when (58) and (59) are valid, for then 











a 

| ho =< >, anh, ~- €9 — 0, (89) 
k=—N 
N 

ha — >, ath, — et || 0, (90) 
k=l 











by the definition of convergence of the infinite sums in (58)-(59). 
Each succeeding equalizer defined by (89)—(90) is obtained by adding 
an additional tap, without changing the other tap-gains. As observed 
by Doob (Ref. 3, p. 564), a convergent sum of the form of (58)—(59) 
does not always exist; the following theorem gives sufficient conditions 
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for the validity of (58)—(59) which are generally satisfied in practical 
problems: 


Theorem 41: If there exist constants K, and Kz, 0 < Ki S Ka, such 
that Ki S R(w) S Ka, |w| < 1/T, then convergent expansions of ey and 
eo of the form of (68)-(69) exist. Furthermore, the coefficients of the 
EXPANSIONS are unique. 


This theorem is proven in Appendix B. The question of uniqueness 
of the tap-gains of the DFE is one which was not answered by Price.® 

Finally, the white output noise property of the MFTF DFE also 
extends to a finite MFTF DFE in the following sense: If the reception 
of (1) extends from N, to Ne, where Ne (but not necessarily N1) is 
finite, then the DFE defined by 


ex =h, — Phi, M(hn, k +185 mS N2)] 


will have white output noise samples. This fact is easily verified from 
the same containment of subspaces that was used in the proof for the 
infinite case. 


IV. EXTENSION TO NONSTATIONARY NOISE AND CHANNEL 


The previous sections have considered only the case where the 
additive noise is white. The extension to colored Gaussian noise can 
be handled in a straightforward fashion with the addition of a whiten- 
ing filter. In this section we will generalize the ZFE and DFE to the 
case of arbitrary nonstationary second-order Gaussian noise (which 
includes colored Gaussian noise as a special case) using the techniques 
of reproducing kernel Hilbert space (RKHS)." Although the cases 
for which the corresponding RKHS can be characterized explicitly 
correspond generally to those cases which can be handled by other 
techniques, the RKHS approach does allow us to treat all cases 
simultaneously and concisely. In addition, it enables us to generalize 
simultaneously to an arbitrary nonstationary channel (to be precise, a 
channel which is changing in time in a deterministic and known fashion) 
with no additional complications. Perhaps the most interesting out- 
come of this effort will be the observation that the DFE white output 
noise property (discussed in Section 3.3) remains valid in this general 
case. The result is an interesting generalization of Forney’s whitened 
matched filter.” 


' Theorem 4 remains valid under the weaker hypothesis that 0 < ess inf R(w) 
and ess sup R(w) < o. 
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To this end, modify (1) to 


= Buhn(t) + (0, (91) 


where, as before, N; and Ne can be infinite. The noise will be assumed 
to be Gaussian with arbitrary autocorrelation 


K(t, s) = E[n(t)n(s) ]. (92) 


The subscript m on h,»(t) indicates that the received pulses need not 
be translates of the same elementary waveform. The reception will be 
termed channel stationary when 


and noise stationary when 
Kit, s) = K(t — s). 


We denote by L2(n) the subspace of the Hilbert space of square 
integrable random variables spanned by n(#), — ~» <t< o. This 
subspace is entirely analogous to M(X;, — ~ <k< @) defined 
earlier, except that the underlying parameter ¢ is continuous. The 
following lemma is applicable 4 


Lemma 2: Let H(K) consist of all functions g(-) of the form 


g(-) = E{n(-)U] (93) 
for some U € Le(n). Then H(K) is a Hilbert space with inner product 
(9, 9)u(K) = E|U|?. (94) 


The mapping y: L2(n) > H(K) defined by (93) is a congruence 
which maps n(¢) into K(-, 2). 

The Hilbert space H(K) defined by Lemma 2 is known as the re- 
producing kernel Hilbert space with reproducing kernel K. It is 
straightforward to show from (93) and (94) that H(K) has the 
properties 

K(.,) €H(K), -—% <t<, (95) 


(9(-), KC, 0)am =9@0, 9 © HK). (96) 


It can be shown" that for any symmetric positive-definite kernel K 
there exists a unique Hilbert space satisfying (95)—(96). 
The inverse of g(-) under y is usually given the suggestive notation 


(9, n)acKy = W*(g) (97) 
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even though n € H(K) with probability one and therefore (97) can- 
not be given an interpretation as an inner product. 

It will be assumed that h,,(¢) GC H(K), since otherwise the detec- 
tion problem is singular.t!! In nonstationary noise the space H(K) 
takes the place of Lz in the earlier white noise problem. Accordingly, 
we restrict the class of filters under consideration to H(K) inner 
products with elements of H(K). Thus, a filter can be written in the 
form 


Ne 
(9, T)H(K) = RL hm)H(K) + (9, 2)H(K), (98) 


where the noise term in (98) assumes the special meaning of (97). 
Analogously to (15), we define the pulse autocorrelation 


Rim, n) = (hm, hn)eck)- (99) 


When the reception is noise and channel stationary, R(m, n) is a func- 
tion of the difference of its arguments, as in (15). In general, however, 
it is an arbitrary symmetric positive definite function defined for 
Nism,n& N.2} 

In the white noise case, we saw that the subspace of Le spanned by 
translates of h(t) was congruent to the subspace of second-order 
random variables spanned by a wide-sense stationary random process. 
In the nonstationary noise case, the subspace of H(K) spanned by 
hm, Ni S mS No, 1s congruent to the subspace of the second-order 
random variables spanned by a possibly nonstationary second-order 
random process. In the white noise case the theory of minimum mean- 
square error estimation of a wide-sense stationary random process was 
relevant; in the present case the random process becomes nonstation- 
ary. As before, the ZFE and DFE have interpretations as interpolation 
and prediction errors of the corresponding random process with auto- 
correlation R(m, n). However, rather than pursue these correspond- 
ences further (in view of our results for the white noise case they are 
obvious), we will directly pursue the theory of the ZFE and DFE 
for the detection of Bz, Ni S m S No, from r(é) in (91). 


T A singular detection problem is one in which a decision can be made which is 
correct with probability one. 


* The positive definite property follows from the inequality 


0<| 








N 
DD Aman (kim,Kn). 


N 
2 Om) km 
m=1 1 n=l 





: N 
H(K) = 2B 
m= 
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The theory of Section 3.1 remains valid if the subspaces M (hn, mE) 
are considered as subspaces of H(K) rather than L2.' As before, the 
condition which is necessary and sufficient for the existence of a ZFE 
or DFE is that 

hr & M (hn; ME I). 
The analogs of the MFTF versions of the DFE and ZFE are the ele- 
ments given by (54) and (55), except that now we must work with e, 
and e; instead of ep and et (e, is no longer necessarily simply a time 
translate of éo, etc.). A derivation similar to that given in Section 3.3 
establishes that e, and ef maximize the S/N ratio as before. In par- 
ticular, when the filter of (91) is restricted to be a ZFE, (91) becomes 


(9, Tuck) = Bitg, he)acxy + (9, 1) aK) (100) 
and the S/N ratio is proportional to 
2 
S/N x Mo helie < (64, e0) cx) (101) 
(9, 9)H(R) 
since the variance of the noise term in (100) is, from (97), 
E|\(g, n)au|? = El¥"@) |? 
= (9; 9) H(K) 


through the congruence established in Lemma 2. Equation (101) 
demonstrates that the MF TF ZFE maximizes the S/N ratio, and the 
same result follows for the DFE by the same method. 

A general equation can be given for the projection element required 
for the MFTF. This equation is entirely analogous to a result of 
Parzen" for stochastic estimation. To this end we require a lemma 
which is a restatement of Lemma 2: 


Lemma 8: Let H(R) consist of all functions f(m), m € I, of the form 
f(m) = (hn, F)acrym EL (102) 
for some F © M(hm, m € I). Then H(R) ts the REHS with reproducing 
kernel R(m, n), m,n € I, and has inner product 
(f, foacay = (PF, F)ace). (103) 


The mapping ¢: M(hn, mC I) — A(R) defined by (102) is a con- 
gruence which maps hm into R(-,m). 


T We use J as a set of indices to avoid repeating the equations twice. For the ZFE, 
I = (Mi, k — 1)U[k + 1, No] and for the DFE I = [k + 1, Ne]. For the infinite 
case, Ne = — Ni = o~. The digit B; is being detected. 
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The reader might find it instructive to verify from (102)—(103) 
that the RKHS properties hold for H(R), 


R(-,n) € H(R), (104) 


(f(-), RC, n)) ace) = f(n), (105) 
where f(-) € A(R). 
The problem we want to attack is finding the projection P of some 


vector Q on M(hm, m € I) (later we will let Q = hx). From (8) we 
have 


(Q — P,hm)xx) = 9, mel (106) 
or 

(P,hm)n(K) = po(m), mel, (107) 
where 

pa(m) = (Q, hm)acK), mc I; (108) 


In (107), pg(m) is a known function and P is to be determined. Assum- 
ing for the moment that pg € H(R), from Lemma 3 we see that pe 
is the image of P under the congruence ¢, and hence 
P= ¢ (pq), (109) 

which is the solution we desire. Using the congruence properties of ¢, 
the length of Q — P is 
lQ — o(pa)llzu = [lla — 2(Q, (Pe) wcK) + IO (Pa) llFram 

= Madlbezes - [loll area (110) 


Establishing that in fact pg € H(A) is straightforward. Note that 
pa(m) = (Q, hm)x(K) 


= (Q — P, hm)u(Ky) + (P, hm) H(K) 
_ CF him) H(K)) (111) 


which implies that pg © H(R) by Lemma 8 since P € M(hn, m € I). 
Replacing Q by h; in (109), we get the desired projection 
Plhx, M (hm, mE I) ] = og LR(k, -)d (112) 


The ZFE and DFE are obtained by letting J equal the appropriate 
set. The S/N ratios of the receivers are proportional to, from (101) 
and (110), 


S/N « | hellzra 7 || (k, +) Er cay (113) 
The RKHS approach has reduced the problem to that of finding 
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RKHS inner products. In some cases these inner products can be 
explicitly characterized, while in all others they can be determined 
by convergent iterative techniques." 

We can also quickly show that the DFE white output noise property 
discussed in Section 3.3 generalizes. From (98), the noise samples at 
the filter output are 

ne = (et, 2) (K) 
= we) (114) 


by definition. From (114) and Lemma 2, 
E(nyne) = EL (e7 yer) J 


= (oe, ef a(x) 
= Q, 7H#k (115) 


by the same reasoning as before. 

Finally, it is instructive to demonstrate that this RKHS formu- 
lation reduces to the whitening filter approach when the reception is 
noise and channel stationary. Assume that 


K(t, s) = - [ * pie t—9N (w)deo , (116) 


where N(w) is uniformly bounded and never vanishes. Under these 
conditions we claim that H(kK) consists of all integrable g(t) with 
Fourier transforms G(w) which satisfy 


lolli = 52 [16 i yy de (117) 





To verify this, properties (95)—(96) must be checked. Equation (95) 
is valid since NV (w) is integrable, while (96) follows from 


(g(-), K(-, ))2cx) = = is O(a) Le-*N (oT FE a 


= 5 is G(w)et!? dw 
= g(t), (118) 


where (*) denotes complex conjugation. From (117), the H(K) inner 
product consists of a filter with frequency response N-!(w) (which is 
the whitening filter) followed by an ordinary Le inner product, and is 
therefore consistent with the whitening filter formulation. 


1516 THE BELL SYSTEM TECHNICAL JOURNAL, NOVEMBER 1973 


V. CONCLUSIONS 


This paper has presented a unified and rather thorough treatment 
of the ZFE and DFE. In a companion paper,’ the geometric model of 
intersymbol interference developed here will be used to study the 
minimum distance problem encountered in the performance analysis 
of the maximum likelihood detector!” and in evaluating a lower bound 
on the performance of any receiver." It is shown there that a canonical 
relationship exists between the minimum distance and the performance 
and tap-gains of the MFTF DFE. 

No performance example comparing the DFE and ZFE on a channel 
of practical interest has been given in this paper in order that the 
maximum likelihood detector may enter into the comparison. In Ref. 7 
the performance of three receivers is calculated for a channel whose 
loss in dB increases as the square-root of frequency. This channel is 
an excellent model of coaxial cable and some types of wire-pairs. 
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APPENDIX A 


The purpose of this appendix is to derive an approximation to the 
Fourier coefficients of (48) in terms of discrete Fourier transform 
(DFT), which can be efficiently evaluated using the FFT algorithm. 

Define a normalized function 


R( =" \) 
F(A) = log a (119) 
so that 
t= 5 © g-inted F()dd- (120) 
Approximating the integral by a summation, 
; 1 Ne=1 1 
tn = 
k=0 


k; : 
aN > F (» + N ae: 5) e— in2a (o+k/ N—}4) 


N-1 
1S F 
k=0 


1 


5 en ante OD (r a we = 5) e~Rr(knIN) — (J21) 


where the sum on the right is a discrete Fourier transform. 
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In order to determine the effect of this approximation, substitute 


4F (A) = 2, fe Tk2md (122) 


into the approximation equation (121) to yield 


09 1 N-l 
Pn = Yt xe YL edlmn)24 Qotk/N—-}) 
mao N k=0 


Thus, the approximation of (121) yields the desired Fourier coefficient 
plus the sum of alias terms. N must be larger than the number of co- 
efficients to be evaluated and large enough that the alias terms ra;in 
are small. In practice, V & 5,000 can be achieved with modest amounts 
of computer time using the FFT algorithm. 


APPENDIX B 
Proofs of Theorems 


Proof of Lemma 1: Since |leg"||? = ||eo||? it suffices to show that |les|| > 0 
implies that {hn, — © < m < o} is linearly independent set. To this 
end, assume that 


N 2 
> a mNiem 


m=l1 














= 0, ki < ke << +++ < ky: (124) 


To show that a, = 0, assume to the contrary that a, ~ 0 and note 
that 
2 


= Jailer] > O- (125) 


0 = [a |? 


N Qm 
hey dX, ay Mim 


1 














This contradiction establishes that a; = 0. Continuing by induction 
in the same fashion, it can be shown thate, = 0,lsimsN. 

To show that the second condition of Lemma 1 implies linear 
independence, we use a proof similar to Tuft’s.% By the congruence of 
(22), (124) is equivalent to 


ar {T 

l 

which implies that the integrand is zero almost everywhere on [a, 0 ]. 
This is impossible unless am = 0,15 m SN, since otherwise 


2 
R(w)dw = 0, 


N 
m=1 








N 2 


m=1 
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has at most a finite number of algebraic zeros on [a, b] and R(w) is 
strictly positive. 


Proof of Theorem 3: We will prove the result for the ZFE; the proof 
for the DFE is identical. Since for N < M 
the inequality 

lleoll S lleo(2Z)|] S lleo(¥)|| 


follows. Hence |l/eo(J/)||? must approach a limit, 
lim |leo(V)I| 2 lleoll- 
Denote by the shortened notation P the projection of ho on M (hx, k¥0) 


(so that eo = ho — P). Since P C M(hy, k ¥ 0), there exists a se- 
quence yn © M(hi, |k| S n, k # 0) such that y, — P and we have 


I[ho — yall? = lleol]? + ||P — vall?. 
For any ¢« > 0, there exists an N(e) such that 
[ko — yall? S |leoll? + ¢ 
for n = N(e), and since ||eo(n)||? S ||ho — yxl|? we have 
lleoll? S lleo(n)|l? S [leoll? + ¢, 


which establishes that |/eo(n)|| — |leol|. The remainder of the proof 
follows that of the projection theorem. By the parallelogram law, 


lleo(N) — ell? = 2|leo(N)||? + 2lleo||? — lle(N) + eoll?, 
but defining P(N) = Plho, M(hz, |k| S$ N,k 4 0)] 
l|eo(N) + eol]? = [ho — P(N) + ho — Pll? 


; ae 
an a 


= 4 > Aled, 





we have 
leo) — ell? S 2[|leo(W)||? — |leo||? ] — 0. 


Proof of Theorem 4: From (22) we have 


2 1 n{T N 
7 Qa L, 


m=l1 


N 2 
| DD Bulk, R(w)dw- 
m=1 
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A standard result of Toeplitz theory asserts that 


1 


T 


| > nl] ess inf R(w) < 


2 


N 
2, Bmht 














1{ 2X 
vn | >; m4 ess sup R(w)- 
m=1 


The conclusions of the theorem then follow from Theorem 5.17.18 of 
Ref. 10. 
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A Geometric Theory of Intersymbol 
Interference 


Part If: Performance of the Maximum 
Likelihood Detector 


By D. G. MESSERSCHMITT 
(Manuscript received May 24, 1973) 


Inacompanion paper,' a geometric approach to the study of intersymbol 
interference was introduced. In the present paper this approach is applied 
to the performance analysis of the Viterbi algorithm maximum likelihood 
detector (MLD) of Forney.2-* It 1s shown that a canonical relationship 
exists between the minimum distance, which Forney has shown determines 
the performance of the MLD, and the performance and tap-gains of the 
decision-feedback equalizer (DFE). Upper and lower bounds on the 
minimum distance are derived, as 1s an iterative technique for computing 
at exactly. 

The performances of the MLD, DFE, and zero-forcing equalizer (ZFE) 
are compared on the Vf channel representative of coaxial cables and some 
wire pairs. One important conclusion is that, previous statements not- 
withstanding,?* even the MLD experiences a substantial penalty in S/N 


ratio relative to the zsolated pulse bound on this channel of practical 
enterest. 


I. INTRODUCTION 


Forney? has detailed the Viterbi algorithm version of the maximum 
likelihood detector (MLD) of digital sequences in the presence of 
intersymbol interference. He asserts that the probability of bit error 
of the MLD in additive white Gaussian noise can be bounded at high 
S/N ratios in the form 


K1Q ain ) <P, < K.Q cae) (1) 
20 20 


where Ky, and A, are constants, Q is the Gaussian distribution 
1521 
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function, 


Q(x) = THR dy , (2) 


1 f a 
a € 
VQr Je 
dmin 18s the minimum distance between any two transmitted signals 
(it will be defined more fully in Section 2.2), and o? is the noise variance. 
For comparison purposes, the probability of error for a matched filter 
receiver in the absence of intersymbol interference is 


=) | (3) 


P,.=Q ( a 
where Fo is the energy of an isolated pulse [ (1) reduces to (8) in this 
case |. 

Forney also asserts that the lower bound of (1) is also a lower bound 
on the error probability of any receiver.4 Thus, the MLD achieves, 
within the multiplicative constant K,/K ,, the minimum probability 
of error attainable by any receiver at high S/N ratios, and, in a very 
fundamental sense, the quantity 


drnin/ Ro 


is a measure of the effective decrease in the S/N ratio (relative to the 
detection of an isolated pulse) resulting from intersymbol interference. 

The determination of the quantity d?,, (known as the ‘minimum 
distance problem’’) is therefore a very important one for, even if the 
implementation of the MLD is not contemplated on a particular 
channel, d?2,, is a measure of the potential performance which can be 
obtained using receivers of arbitrary complexity. Unfortunately, on 
channels with severe intersymbol interference, the exact analytical 
determination of d?2,,, does not appear feasible because of the nonlinear 
nature of the problem. 

The minimum distance can be determined numerically by the 
“brute force” technique of calculating a sequence of converging upper 
bounds. A shortcoming of this method is that it gives no assurance 
as to when convergence to the desired accuracy has occurred. In 
addition, it gives no insight into the nature of d?,,, and its relationship 
to the intersymbol interference or to the performances of other 
recelvers. 

In this paper, we attack the minimum distance problem using a 
geometric theory of intersymbol interference developed in companion 
papers.!® A canonical relationship will be shown between d3,,, and the 
decision-feedback equalizer (DFE). This relationship will be exploited 
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to derive simple lower and upper bounds on a2, 1n terms of the tap- 
gains of the DFE transversal filter and the S/N ratio performance of 
the DFE. In addition, an iterative procedure will be derived for the 
calculation of d?,, to any desired accuracy using a sequence of con- 
verging upper and lower bounds on d?2,,. The lower bounds give us a 
measure of the degree of convergence and enable us to terminate the 
calculation when the desired accuracy is assured. 

After consideration of the minimum distance problem in Section II, 
the performance of the zero-forcing equalizer (ZFE), DFE, and MLD 
is compared on a channel of practical interest in Section III. 


II. PERFORMANCE OF THE MLD 


The minimum distance problem will now receive consideration. The 
first step 1s to briefly review the notation of a companion paper.! 


2.1 Notation 


The reception from a PAM communication channel takes the form 


r(t) = 2X Byhtt — kT) + n(d), (4) 


where each B; assumes one of a finite number of predetermined values 
(the data being transmitted), h(¢) is square-integrable (element of 
L,),* and n(t) is white Gaussian noise. 

When we denote h(t — kT) as an element of Le by hz, M(hi, k € J) 
is the smallest closed linear subspace of Le containing all finite linear 
combinations of elements of the set {h,, k € I}. The projection of a 
vector x on M(hz, k € I) is denoted by Plz, M(hi, k € I) ]. The for- 
ward matched-filter transversal-filter combination of the DFE cor- 
responds to the Lz inner product of the reception r(t) with the element 


et 2h, — Plhs, M(hm, m > k) | (5) 
and is orthogonal to the subspace M (hn, m > k). The quantity 


leg" ||? 
Ryo ’ 
where 


A 
Ry = (hm, hm+x) (6) 
* We denote by Lz the space of square integrable waveforms with inner product 


ay) = [" e@yod 


and norm ||z||? = (2,2). 
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is the effective decrease in S/N ratio relative to an isolated pulse for 
the DFE. Thus, |leg"||? plays the same role for the DFE as d?,, plays 
for the MLD. 

The sequence of vectors {wz © ej/|le¢||} is an orthonormal sequence 
in Le, and h, has the orthogonal expansion 


= » C'mW m+n (7) 


where the coefficients {C,,} can be determined by the method of 
Ref. 1 for channels with either a rational or nonrational spectrum. 
In particular, we have 

Co = lec ||. (8) 


Of course, it is apparent that (7) is valid only as long as |leg|| > 0, 
which is true if and only if a DFE exists. 


2.2 Interpretation of the Minimum Distance 


The MLD described by Forney? consists of a combination of a 
matched filter followed by a causal or anticausal transversal filter, 
the combination of which he calls a “whitened matched filter,” 
followed by a dynamic programming algorithm known as the Viterbi 
algorithm.’ The whitened matched filter forms a sequence of sufficient 
statistics for the detection of the data digits and has independent noise 
samples at the output. As pointed out by Price,* the anticausal 
whitened matched filter is identical to the forward linear filter portion 
of the DFE. 

The signal at the output of the whitened matched filter (or DFE 
forward filter) is! 


rT, = C2B;, + Le CoC mB r—m + Nk (9) 
where n; 18 a noise sample. The DFE forms the quantity 
1 = fp = CoC 'm Bice (10) 


and applies it to a decision threshold to determine the estimated digit 
B,. The MLD detector, on the other hand, assumes that the sum in 
(9) is truncated to M terms and determines the sequence {B;} so as 
to minimize 


p> TC. — a CoC nBin} (11) 
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Thus, the two receivers perform similar functions on the same sufficient 
statistics r,, the major differences being the greater complexity of the 
MLD and the susceptibility of the DFE to decision errors. We will 
now demonstrate the less obvious conclusion that the performance of 
the MLD 1s closely related to the DFE as well. 

The minimum distance, d?2,,, is defined as? 














A v : 
din = inf || >> enhall, (12) 
«0 ~0 n=0 
where the infimum is over all error sequences (€9, ---, ev) and all N.* 


Each e, assumes the value +1, —1, or zero (for simplicity, the binary 
case with B, = 1 or O is considered). Thus, dmin is the minimum 
distance in Le between two signals in the signal set. It is apparent that 


7 Ro, (13) 


since /» corresponds to en = 0, n > 0. Thus, dy,/Ro, which is the 
S/N ratio penalty, is a number between zero and unity as it should be. 

It is apparent in (12) that without loss of generality we can choose 
é9 = 1 and write 


Qe = nt 





N 2 
ho + L niin 








(14) 


The sum in (14) is an element of M(h,, k = 1), and the minimization 
in (14) is an attempt to find the element of M(h;, k = 1) with manifold 
coefficients (+1, —1, 0) which is closest (in £2 metric) to ho. We know 
that the closest element without the restriction in coefficients is the 
projection of ho on M(hy, k 2 1), P[ho, M(hi, k 2 1) ]. Thus, in- 
tuitively, d2;, is determined by how closely the projection can be 
approximated by an element with restricted manifold coefficients. To 
formalize this intuition, add and subtract the projection from (14) and 
utilize (5), 


2 








N 
eg + Plho, M(hi,k 2 1)] + X Enkin 








*, (8) 


leo ||? + inf 





N 
PL, M (hay k = I+ E enh 








where the fact that eo” is orthogonal to M (hx, k > 0) has been used 
to eliminate the cross-product in (15). The most immediate conse- 
quence of (15) is that 


rnin = = les ?. (16) 


“In most cases of interest, the infimum will be achieved for finite N. 
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We have thus succeeded in proving formally what should be obvious 
from considerations of the relative complexity of the two receivers: 
The effective S/N ratio of the MLD always exceeds that of the DFE 
(and hence ZFE!).* The second consequence of (15) is the formaliza- 
tion of our intuition through the assertion that the amount by which 
the S/N ratio of the MLD exceeds that of the DFE is governed by the 
coarseness of the best approximation to the projection by the element 
with restricted coefficients: The poorer the approximation, the better 
the S/N ratio of the MLD. 
Writing the projection in the form 


Plho, M(ha, k > 0)] = — ¥ athe, (17) 
m=l1 


we note that the a, are the tap-gains of the DFE forward transversal 
filter, and rewrite (15) as! 

2 

dian = |leg ||? + inf : 


(18) 








y (En — Ot )hn 
n=l 


Equation (18) shows the fundamental relationship between the 
minimum distance, the effective S/N ratio of the DFE (in the form 
of |leo||?), and the tap-gains of the DFE transversal filter. In particular, 
we can assert that d2, = |/eo ||? if and only if the tap-gains are all +1, 
—], or zero. 








2.3 Bounds on the Minimum Distance 


Equation (18) can be used to derive bounds on dz, in terms of the 
DFE tap-gains. From the identity? 


N 2 


























N En — ar 
De (én — at )An| |? = (ex — at)?)|he + 2 ——Fha|| , (19) 
n=1 as) €k — Ay 
we immediately get the bounds 
y 2 ((a—aryller|?, k=l 
DX (en — at )hn|| 2 (20) 
n=1 














(e, — ag )Pllel?,  & > 4, 


*'We are tempted to argue that (16) is implied by the assertion in Ref. 2 
that the MLD achieves the lowest effective S/N ratio of any receiver. However, 
that is not the case, because of the effect of decision errors on the DFE. The effective 
S/N ratio of the DFE could be higher than that of the MLD, and yet the DFE could 
have at the same time a higher error probability because of error propagation. 

t We have taken the liberty of writing a sum over infinite error sequences, where 
it is understood that the infimum is only over error sequences with a finite number 
of nonzero terms. 

*In (19) it is assumed that (e, — at) #0. When «& — af =0, (20) is trivially 
satisfied. 
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since 
N (én — ay) 
2 (ex — aj) Ms 


is an element of M(hn, m #¥ k). In (20), eo is the ZFE filter defined in 
Ref. 1, 


é9 & ho — P[ho, M (hx ,k ¥ 0). (21) 
In addition, if we define \min(V) and Amax(NV) as the minimum and 
maximum eigenvalues of the correlation matrix 

Ry = [Rim—n| 1 < m,n <N, 
then we can assert that 
2 


N N 
Amin(NV) a (én —Q;)? S | Pa (€n — a )hn 


n 








N 

S Amax(V) Do (en — ar)? (22) 
n=1 

A standard Toeplitz form result’ asserts that* 


(itn: Men) = eet 


N— 0 £ 


| 


fi tev 7 ss nrteGa: 
Noo 


Applying (18), (20), and (22), we get three lower and one upper bound 
on dz, 1n terms of the tap coefficients of the DIE, 


lec |? min (a, — aj")? 


dim = |led ||? + 4 |leol|? min (e. — af)?, k> I 
N 
fesse RG) lim: ain, 3 ea? 
T No €1,°°*',eN n=l 
1 : . N’ 
din & |ler ||? + ir fess sup R(w)} lim min > (en, — az)? (23) 
No €1,¢°°,en n=l 


In addition, an upper bound can be obtained by substituting any error 
sequence into (18); a reasonable choice is 


has ax < aa 
eg = 0, —i<ajy < $. (24) 
—1, af >3- 


*For all practical purposes, “‘ess inf’? and “ess sup” can be replaced by ‘‘min’’ 
and ‘‘max,”’ respectively. 
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These five bounds can be useful in estimating the penalty in S/N 
ratio for the MLD. They all require the existence of a DFE and 
require that the projection can be written as the convergent sum of 
(17).* The second and third bounds of (23) are an improvement on 
(16) only when the increasingly stringent requirements that a ZFE 
exist (||€o|| > 0) and R(w) be uniformly bounded away from zero 
(almost everywhere) are imposed. The requirement of the upper 
bound of (238) that R(w) be uniformly upper bounded (almost every- 
where) will generally be satisfied in practice. All the bounds require 
a pointwise minimization over error sequences, a task much simpler 
than minimizing (12) directly. 

As a simple application of these bounds, consider the exponential 
autocorrelation 


Ry, = Al, 0<A <1. (25) 
Then we have!? 
i. 0<AS} 
rain om 
am 1 
2(1 as 2 < A <1 (26) 
Jeol? = (1 —- A?)/(1 + A?) 
lec ||? = 1 — A? 
ai = — A, az = 0, k> 1. 
The first and third bounds of (23) become 
1— At, 0<AS}3 
mie = (27) 
(1 — A®)(2 + A? — 2A), 3<A<1 
1 — 2A?/(1 + A), 0< ASH 
diin 2 (28) 
21S A Ayer A), 2d 
and the upper bound of (23) becomes 
2A3 : 
f 1A } O< A = 2 
Cnt = (29) 


| ea-4), F<AX<1. 


These bounds are plotted in Fig. 1. The upper bound of (24) is equal 
to din and is not plotted. 


*If the projection of ho on P(hx, k = 1) cannot be written in the form of (17), 
the bounds of (22) to (24) can be fixed up by considering the projection on P (hi, 
1 sk SN) and taking limits as N — . The tap-gains will then be a function of N, 
and the process will be more difficult. 
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——— UPPER BOUND 
EQ. (29) 


LOWER BOUNDS 
EQ.(27)-~ 
EQ.(28)-—-— 


| egt |?-— 





0 0.25 0.50 0.75 1.00 1.25 
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Fig. 1—Bounds on d?,,, for an exponential autocorrelation. 


The bounds just determined have the disadvantages that (7) they 
require calculation of the DFE tap coefficients and (77) they do not 
give precise results on d2;,. The exact value of d?2,, can be determined 
numerically by the direct minimization of (12); by letting N —- 
while exhaustively minimizing over error sequences, we get a sequence 
of upper bounds on d?2,, which approach d2,;, monotonically. The 
obvious difficulty with this method is that the number of error se- 
quences which must be checked grows as 3%, and the computational 
effort soon becomes unreasonable. What happens in practice is that 
the true minimum is achieved for a finite (and small) N. However, 
unless we have some method of determining when the true minimum 
is reached, there must always remain a degree of uncertainty as to 
whether the true minimum has been reached. 

Our approach to this computational problem will be to derive a 
sequence of lower bounds on d?,, which also approach d?,, monotoni- 
cally. We can then halt the process at a value of N where the upper and 
lower bounds are close enough to ensure knowledge of d2,, within the 
desired accuracy. To this end, we will utilize the orthogonal expansion 
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of (7). Substituting (7) into the sum of (12), 


oo 


3 Enln = 9 En >. C Wain 
n=0 n=0 m=0 
= 2 Ban, (30) 
where 
Bm = ys €xC'm—k (31) 


k=0 


Then, because the {w,} are orthonormal, 








) 2 oe) 
L eaball = D Be (32) 








It appears that we may have made life more difficult for ourselves, 
because even when we substitute a finite sum on the left of (82) we 
must still evaluate an infinite sum on the right. However, note that 
since the terms in the sum are positive, 


> Enln 


n=0 





2 N 
ee. (33) 











where the sum on the right is always finite and is in terms of a finite 
length error sequence (é€o, ---, ew). Hence, 


@,, = min > 6 (34) 
ie a n=0 
and, furthermore, the right side of (34) approaches the left side 
monotonically as N > o«. 

The minimization of (34) is no more or less difficult to perform than 
that of the direct minimization of (12). It does require the existence 
of a DFE and evaluation of the coefficients {C,,}. A reasonable pro- 
cedure is, at each stage of N, to minimize the right side of (34) to 
obtain a lower bound on d2,,, and substitute the minimizing sequence 
into (12) to obtain the upper bound* on d?2,,. When the lower and 
upper bounds are sufficiently close, the process can be terminated. 


* Note that any sequence substituted into (12) yields an upper bound on d?,,, 
and the one which minimizes (34) is as good as any. On the other hand, only the 
sequence which minimizes (34) yields a valid lower bound, so it must be minimized. 
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The minimization of (34) can be assisted slightly by dynamic 
programming. Defining 


N 
fnu—m(é1; ees €m—1) = min py Bas (35) 
we note that 
N 
min ¥ 6% = min [fya(a) + 62) (36) 
with a recursion relation for fy_m (€1, °°*, €m—1), 


; N 
min ) &B; 


€m-1,°°*,EN n=m—1 


Fu—mt1(e1, ne Em—2) 


et min 3 p+ Bi -| 


€m—1 ems **EN N=™M 


= min [fy—m(e1-++ém—1) + Bm-i]. (87) 
Because there is no possibility of using forward dynamic programming 
in this case, the savings in computation for this method is not too 
spectacular. Each 8, must still be evaluated for 3% error sequences; 
the savings is in eliminating the need for summing 6? for most of the 
combinations of 3% error sequences. 

We note in passing that using the FFT algorithm to reduce the 
computational effort in the convolutional sum of (31) is a possibility. 
However, the 3% sequences for which it must be evaluated becomes a 
limiting factor long before the savings of that method becomes 
substantial. 

In the foregoing discussion, the existence of a DFE has been re- 
quired [that is, |le|| > 0 or equivalently log R(w) is integrable, where 
FR (w) is the equivalent power spectrum of the channel! |. When log R(w) 
is not integrable (as when it vanishes on an interval), there does not 
appear to exist an expansion of the type (31) to (82). What can be 
done is to use the Gram-Schmidt expansion of the form 


hin = = (hiny Wr) Why (38) 


where w,; is the orthonormal sequence obtained from {h;} by the usual 
Gram-Schmidt orthonogalization procedure. This expansion merely 
requires that {h;} be linearly independent, which is guaranteed by the 
existence of an interval on which R(w) does not vanish.! From (88), it 
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follows that 


DL Emin = DY em Dy (hm, Wr )Ws 
m=0 m=0 k=0 


= > Bur (39) 
k=0 
i= > dallas Abi): (40) 


The key point is that the summation in (40) is infinite, so that evalua- 
tion of the lower bound of (84) is now necessarily over infinite error 
sequences. The finite sum in (31) results from the form of the expan- 
sion (7) in which h, is expanded in terms of all future w,’s, and this 
expansion is in turn dependent on h, not being an element of 
M(hi, k > n). Thus, when a DFE does not exist there appears to be 
no alternative to evaluating a sequence of upper bounds to dij, ob- 
tained by a finite sum approximation without the benefit of lower 
bounds to measure the degree of convergence. 


Iii. THE PERFORMANCE OF THREE RECEIVERS ON THE vf CHANNEL 


Results of a calculation of the performance of the MLD, DFE, and 
ZFE will now be reported for the Vf channel, for which the attenuation 
in decibels increases as the square root of frequency. The vf channel 
is a good approximation to coaxial cable, as well as to some cables 
consisting of wire pairs, and for this reason it is of great practical 
interest. 

Many present high-speed digital transmission systems use some 
form of linear equalization, and their performance will be reasonably 
well approximated by that of the ZFE. Thus, the comparison between 
the ZFE and the MLD gives us an indication of the size of the gap in 
performance between common transmission systems in use today and 
what could theoretically be achieved by much more complex receiver 
designs.* The comparison with the DFE is much less interesting, be- 
cause the susceptibility of the DFE to decision errors is not included 
in the present analysis and, as will be shown shortly, is of such a 
magnitude on the Vf channel as to essentially invalidate the perfor- 
mance estimate we calculate. 


*This comparison is, of course, very idealized. The only impairment we consider 
is additive Gaussian noise. 
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Fig. 2—Performance of three receivers on the Vf channel. 


The power spectrum of the Vf channel is given by 
| H(w) |? = 2rK?RoeP* Ve, (41) 


where H(w) is the frequency response of the channel and K is a 
parameter proportional to the line length. The usual convention is to 
designate the loss at the half-baud rate (w = 7/T), 


4 (7) 
PHO 


_,j/£__¥ 
se Jem log e (438) 


The effective penalties in S/N ratio relative to the isolated pulse 
bound can be calculated for the ZFE and DFE using the methods of 
Ref. 1, and for the MLD using the methods developed in Section 
II. The result is shown in Fig. 2 for the range of y of practical interest. 
Most high-speed transmission systems in use today have a y less than 


2 





y = — 10log (dB), (42) 


in which case 
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about 65 dB because of limitations in the maximum gain which can 
be incorporated into a repeater without excessive coupling of the 
output back into the input. 

One interesting feature of Fig. 2 is that even the MLD has a sub- 
stantial S/N ratio penalty (15 dB) on the Vf channel. Thus, Forney’s 
statement’ that on most channels intersymbol interference does not 
have to lead to a significant degradation in performance does not apply 
to channels with very severe intersymbol interference, such as are 
commonly used in high-speed transmission systems. 

The value of d?2;,, valid for Fig. 2, as well as many other examples 
considered by this author and Forney,’ is 


min = 2(Ro — Ry), (44) 


where &; is the autocorrelation of the received pulse.* An approxima- 
tion to (44) valid for large y is derived in Appendix A and plotted in 
Fig. 2 as a dotted line. Approximations to the S/N ratio penalty of 
the ZFE and DFE are also derived in Appendix A and plotted in Fig. 
2. An intuitive interpretation of eq. (44) is given in Appendix B. 

As an illustration of the speed of convergence of (34), the sequence 
of upper and lower bounds is illustrated in Fig. 3 for a Vf channel with 
y = 60 dB. These bounds are within 1 dB for N = 1 and 0.5 dB for 
N = 3. Thus, convergence is very rapid, even for severe intersymbol 
interference. 

A word of caution is in order with respect to the curve for the DFE 
in Fig. 2. This curve does not take into account the effect of decision 
errors on the performance of the receiver. The DFE subtracts, prior to 
the decision threshold on data digit B,, the quantity 


ay bm Bk~m (45) 
where B,_» is the receiver’s previous decision on B,_m and b,, is the 
tap-gain of the DFE feedback filter. The resulting quantity which is 
applied to the threshold is! 


boB, + 3 bn (Dew a By_m) + Nk, (46) 


m=l1 


where n; is a noise sample. Whenever the D,,’s are large with respect 
to bo, a single decision error will likely cause many more errors. The 


* This corresponds to the error sequence (1, —1, 0, 0, -+-) or, in the notation of 
Forney, (1 — D). 
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Fig. 3—Convergence of lower and upper bounds on d2,;, (V f channel with y = 60 dB). 


coefficients of (46), given by (9), are tabulated in Table I for several 
values of y. 

Needless to say, the situation is hopeless for the large 7; the effect 
of a single decision error will be major and will last for a long time. 
Even for y = 20, the reduction in noise margin resulting from a pre- 


TABLE J—CoEFFICIENTS OF THE DFE FEEDBACK FILTER (bm) 


m bm 
y = 20 y = 40 y = 60 
0 1 1 1 
1 0.61 1.4 2.2 
2 0.36 1.3 2.8 
3 0.25 1.1 2.9 
4. 0.18 0.94 2.9 
5 0.14 0.80 2.8 
10 0.06 0.42 1.9 
47 0.006 0.06 0.38 
174 0.001 0.009 0.06 
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vious decision error will be significant for five or ten subsequent . 
decisions. We must conclude, then, that Fig. 2 will not be representa- 
tive of the true performance of the DFE, and further that the DFE | 
may not be a suitable receiver for the Vf channel’. 

In terms of repeater spacing and baud rate, Fig. 1 can be interpreted 
in two ways. If the ZFE is replaced by an MLD, the same level of 
performance can be maintained while either increasing the repeater 
spacing with a constant baud rate or increasing the baud rate with 
the same repeater spacing. To illustrate this, consider the example of a 
ZFE operating at a given level of performance on a Vf channel with 
y = 40 dB. Then y can be increased to 60 dB at the same effective 
S/N ratio. This corresponds to a 50-percent increase in repeater spac- 
ing at a constant baud rate (since y goes up linearly with the repeater 
spacing). However, since the repeater spacing has increased, the trans- 
mitted power must also be increased by 3.5 dB to maintain a constant 
isolated pulse energy at the receiver.' 

If the repeater spacing is held constant, an increase in baud rate by a 
factor of (1.5)?, or 125 percent, will also result in a 50-percent increase 
in y. Here too, the average (but not peak) transmitted power is in- 
creased by 3.5 dB. 

The conclusion of these results is that there is a fairly large gap be- 
tween the performance of linear equalizers and the theoretical limit 
on the Vf channel. It is probably fair to say, however, that practical 
constraints on repeater complexity, speed of operation, and gain makes 
the attainment of a substantial portion of this potential improvement 
on high-speed transmission systems very difficult, at least for the 
present. Such is not the case for low-speed applications, such as voice- 
band data, where the implementation of the MLD can be contemplated 
on the basis of existing technology. 


IV. CONCLUSIONS 


In this paper, the minimum distance measure has been interpreted 
geometrically, related to equalization (the decision-feedback equalizer 
in particular), and bounded in several ways. A practical numerical 
technique has been developed for calculating the minimum distance 
without considering unnecessarily long error sequences. 


* Tomlinson’ has invented a method of avoiding the error propagation problem by 
subtracting out interference from past data digits in the transmitter. 

‘The received pulse energy is proportional to y~?, so that the peak and average 
transmitted power must be increased by 20 log(60/40) = 3.5 dB. 
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Numerical results for the Vf channel reveal that the penalty in S/N 
ratio relative to the isolated pulse bound for the MLD can be sub- 
stantial for this channel, and that the gap in performance between the 
MLD and linear equalization is also substantial. The latter suggests 
that further attempts at finding receivers without the complexity of 
the Viterbi algorithm MLD but which nevertheless improve on the 
performance of linear equalization might well be fruitful. The decision- 
feedback equalizer does not appear to fit this bill because of its serious 
error propagation problem when confronted with intersymbol inter- 
ference as severe as that found on the Vf channel. 


APPENDIX A 


Autocorrelation of the Vf Channel 


From (41), the autocorrelation 1s 





ae [ | H(w) |? cos (wkT) deo 
0 
2 oe) 
_ a i x exp (- ms «) cos (ka?)dz. (47) 


Integrating by parts with u = exp ( — 7 «) and dv = x cos (k2?)dz, 


we get 
AK™Ry f” 2K 
Re = Gone |. exp (— Ta) sin atde, 


which is given in terms of the Fresnel Integral,° 
m= Ja cere (La © eps) |= (er) 
+E-5(By2))@(8))- « 
C(x) = i cos (5r) dy 


S(a) = [sin (52) dy. 


An accurate approximation to R, valid for large y is easily obtained 
from (47) by substituting the first two terms of a Taylor series for 


where 
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COS 2, 
ra) a4 
Ri = B'Ro | x (1 — — ; e8z da 
0 2 
60 
-R (1-5), (49) 
where 
3 = 2K. 
VT 
Hence 
120 
2(Ro — Ri) = er 
and 
— 10 log eo — Ey ~ 40 log y — 56.2. (50) 
0 


Approximations to |leo||? and |leg"||? can also be derived by assuming 
that H(w) = 0, |w| > 7/T, or equivalently that |H(w)|? = R(w). 
The resulting S/N ratio penalties are 

— 10 log |leol|?/Ro = y + 25.15 — 30 log y (51) 
— 10 log |leg||?/Ro = #y + 15.76 — 20 logy. (52) 


Equations (50) to (52) are plotted in Fig. 2 as dotted lines. 
APPENDIX B 


Interpretation of Equation (44) 


It is straightforward to show that whenever 


Ry 


0 


= 0.5 (53) 


we have 
dain S 2(Ro — Ri) S Ro. (54) 
Noting that 
Ri = (ho, hi) = |[holl ||Aall cos 
= Fo cos 0, 


where 6 is the angle between hy and hy, eq. (53) becomes 
@ <= 60°. (55) 


The geometric interpretation of (55) is shown in Fig. 4, where it is 
seen that (54) is satisfied until 6 = 60°, when the triangles become 
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Fig. 4—Geometric interpretation of eq. (44). 


equilateral. As long as (55) is satisfied, ho — hi is a shorter vector than 
ho. 


In the case of the Vf channel, R1/Rp is very close to unity. Thus, 


ho — hiis a very short vector. Although it will certainly not always be 
the case, a plausible explanation for the fact that longer error sequences 
do not yet yield a shorter vector is that the addition of other translates 
of hy, (such as +he) adds further components in other dimensions. 
Presuming that it does not reduce the component in the hy — hi plane, 
it can then only increase the length of the total vector. 
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Adaptive Channel Memory Truncation 
for Maximum Likelihood Sequence 
Estimation 


By D. D. FALCONER and F. R. MAGEE, JR. 
(Manuscript received May 15, 1973) 


Maximum likelihood data sequence estimation, implemented by a 
dynamic programming algorithm known as the Viterbs algorithm (VA), 
as of considerable interest for data transmission in the presence of severe 
entersymbol interference and additive Gaussian noise. Unfortunately, the 
required number of receiver operations per data symbol 1s an exponential 
function of the duration of the channel impulse response, resulting in 
unacceptably large receiver complexity for high-speed PAM data trans- 
- mission on many channels. | 

We propose a linear prefilter to force the overall impulse response of 
the channel/prefilter combination to approximate a desired truncated 
empulse response (DIR) of acceptably short duration. Given the duration 
of the DIR, the prefilter parameters and the DIR itself can be optimized 
adaptively to minimize the mean-square error between the output of the 
prefilter and the desired prefilter output, while constraining the energy in 
the DIR to be fixed. 

In this work we show that the minimum mean-square error can be 
expressed as the minimum eigenvalue of a certain channel-dependent 
matriz, and that the corresponding eigenvector represents the optimum 
DIR. An adaptive algorithm 1s developed and successfully tested. The 
semulations also show that the prefiltering scheme, used together with the 
VA for two different channel models, compares favorably in performance 
with another recently proposed prefiltering scheme. Limiting results for 
the case where the prefilter 1s considered to be of infinite length are obtained; 
it 1s shown that the optimum DIR of length two must be one of two possible 
impulse responses related to the duobinary impulse response. Finally we 
obtain limiting results for the case where the transmitting filter 1s optimized. 

1541 
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I. INTRODUCTION 


Forney! has recently proposed a receiver structure for a communi- 
cation system operating over a known time-dispersive channel with 
little loss in performance due to intersymbol interference by using 
maximum likelihood sequence estimation, or the Viterbi algorithm 
(VA).2 This has resulted in much attention being given to practical 
methods of applying his results. Magee and Proakis’ proposed the use 
of the VA directly in conjunction with a channel estimator. This 
approach can result in a receiver too complex for practical use because 
the complexity of the VA depends exponentially on the duration of the 
channel impulse response. 

In particular, if the impulse response of the channel has an effective 
duration of + seconds and if an L-level PAM system transmits 1/7 
data symbols per second, the number of operations per received symbol 
is proportional to L7/7. For channels such as voiceband telephone 
channels, the bandwidth of which is used efficiently, typical values of 
T/T may be between about 20 and 200, making direct application of 
the VA infeasible. 

Thus, 1t seems clear that effective practical application of the VA 
or of related techniques involves a compromise between optimum 
performance and receiver complexity. The complexity-limiting ap- 
proach we take here is to use a linear prefilter at the receiver to 
“condition” the overall sampled impulse response seen by the VA so 
that it 1s significantly different from zero over only a small number of 
samples, and any remaining intersymbol interference is considered to 
be noise. Additional joint optimization of the transmitting filter is 
also treated, but would be much harder to implement in a real system. 

The simplest example of a prefilter is a linear equalizer, which yields 
an approximate overall impulse response of just one sample. Another 
example of prefiltering for a different purpose is the linear portion of a 
decision-feedback equalizer; in that case the initial sample of the de- 
sired overall impulse response is required to be large relative to the 
additive noise. 

In any application of prefiltering to approximate a desired impulse 
response (DIR), the DIR itself and the prefilter should be chosen to 
minimize the error due to noise and to the difference between the DIR 
and the actual impulse response that 1s achieved. The latter error re- 
sults from intersymbol interference components outside the interval 
accounted for by the DIR samples as well as from errors in approxi- 
mating the DIR inside the time-limited interval. This error could be 
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eliminated by using a zero-forcing criterion at the cost of additive 
noise. 

Qureshi and Newhall‘ have recently proposed a receiver incorporat- 
ing prefiltering with the VA. They use a mean-square error (MSE) 
criterion to force the overall response of the channel plus the linear 
equalizer to approximate a truncated version of the channel pulse re- 
sponse. In order to decode, the VA assumes this truncated response, 
resulting in much simplified processing. There is no effort made in 
Ref. 4 to optimize the desired truncated response. It is the purpose 
of this paper to see how this desired response can be chosen to minimize 
MSE and to show that this receiver structure can be made adaptive. 

In Section II we formulate the MSE-minimization problem, assum- 
ing a fixed number of samples in the DIR and in the impulse response 
of the prefilter. The minimum achievable MSE is the minimum eigen- 
value of a certain channel-dependent matrix. In Section III we indicate 
how the prefilter tap coefficients and the samples of the DIR can be 
determined adaptively by a gradient algorithm based on the MSE 
minimization. Section IV is a study of the limiting situation in which 
the tapped delay line prefilter consists of an infinite number of taps 
and it 1s preceded by a matched filter. Compact expressions for the 
prefilter impulse response, DIR, and minimum MSE are derived, 
which lend further insight. Section V describes the results of computer 
simulations of an adaptive prefilter/VA receiver structure, including 
comparison of the receiver with that described in Ref. 4 and with per- 
formance lower bounds. Section VI presents performance calculations 
for the prefilter/VA system, a decision-feedback equalizer, and a 
linear equalizer for a particular channel. Plots of minimum MSE 
versus bit rate for each of the three types of receiver structures are 
shown. Section VII considers asymptotic transmitter optimization. 


II. OPTIMIZATION OF THE RECEIVER 


The channel model and the preliminary receiver processing are 
shown in Fig. 1. The channel is modeled as a linear continuous filter 
with additive white Gaussian noise. It has been shown by Forney! 
that the channel can then be followed by a matched filter, symbol 
rate sampler, and nolse whitening filter with no loss of information. 
Alternatively, the reader may assume that the channel 1s band- 
limited and symbol rate sampling can be used with no information loss. 

Due to these considerations, the discrete-time model of Fig. 2 was 
adopted with the additional assumption that the channel pulse re- 
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Fig. 1—Channel model and preliminary receiver processing. 
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Fig. 2—Discrete-time channel model. 


sponse is time-limited. The noise sequence {nx} is additive, uncor- 
related, and Gaussian with variance o”. Note that a discrete-time model 
with uncorrelated noise samples also results from the commonly used 
but nonoptimum expedient of passing the received signal through a 
flat Nyquist band-limiting filter prior to sampling. * 

The proposed receiver structure is shown in Fig. 3. The received 
sequence feeds a linear tapped delay line filter whose function is to 
shorten the overall impulse response length. The filter has L(=2M-+1) 
taps which are chosen in the manner to be described later. The output 
of this filter feeds the Viterbi algorithm which detects the information 
sequence. 


* Although a white noise model was used throughout, the correlated noise case 
can be considered in a similar manner. 
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Fig. 3—Receiver structure. 


The Viterbi algorithm makes decisions on the assumption that the 
DIR {q:}%4, is the actual overall channel response. The value of DL 
(the length of the DIR) is much less than 2N + 1 (the length of the 
actual channel response). LL is chosen to make acceptable the com- 
plexity of the Viterbi algorithm while taking a small noise penalty 
in the linear preprocessing.* An error signal is formed by feeding the 
information sequence estimate through the tapped delay line repre- 
senting the desired channel response. This forms the desired truncated 
channel received sequence which is then compared with a delayed 
version of the actual linear prefilter output to form an error sequence. 
It is this error which is to be minimized since it represents a sum of 
the additive noise, and the difference between the desired and actual 
overall impulse responses. 

If the sampled channel impulse response, sequence of information 
symbols, and sequence of uncorrelated noise samples are represented 
respectively by {hi}, {Zi}2_.~, and {ni}2_., then the kth dis- 


* Obviously if LL is allowed to be very large, the DIR can closely approximate a 
delayed version of the original channel impulse response, and there is no significant 
noise penalty, since the prefilter simply approximates a delay tine. 
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crete channel output is 
'. = 2 Ail 1 ++ Nike (1) 


Then if the vectors P+ = (p_m, -+::, Po, --* pm) where + indicates 
transpose and Qt = (qo, -:*, Yu) represent the tap coefficients of the 
prefilter and the DIR respectively, the error in the kth interval is 


M LL 
é, = ny Pivn—1 — 2d Qul .—1. (2) 


In order to simplify the following equations, it is assumed that the 
information sequence is uncorrelated (I,J; = 6,;) and that the in- 
formation sequence estimate equals the information sequence. Sub- 
stituting (1) into (2) and averaging e? we get 


=e = PtAP+ Q*Q — 2P'HQ, (3) 
where 
hu ++ thusu 
H = \h) ---Arz (4) 


hous: shomy uy 


is an L X (LL + 1) matrix and A is an (LZ X L) channel covariance 
matrix with elements a;; = Tir}. 

First, the error is minimized with respect to the prefilter by taking 
the gradient with respect to the taps {p;} and setting it equal to zero. 
The taps {q:} are constrained to be nonzero. 

9" _ 2AP — 2HQ = 0 (5) 
dP 
and therefore 


Poo, = A7HQ. (6) 


The interpretation—thus far—is that some desired (and truncated) 
channel response is chosen; and the linear prefilter taps are chosen to 
force the overall response to this with a minimum MSE. The question 
of what this desired response should be naturally arises. If a fixed 
length is assumed for this desired response, the desired response can 
be optimized in the sense of minimizing MSE. Substituting (6) into 
(3), one obtains 

é = Q+[I — H+A“H]Q, (7) 


where J is the identity matrix. 


ADAPTIVE CHANNEL MEMORY TRUNCATION 1547 


Since e2 = 0, this is a positive definite quadratic form in Q which 
depends only upon Q and the channel characteristics. This can be 
minimized by choosing Q to be the eigenvector with the minimum 
eigenvalue of the matrix [J — H+AH ]. The constraint 


Q*Q=1 (8) 


is necessary to avoid the trivial case of no MSE. The trivial case cor- 
responds, of course, to no transmission through the channel. 

It should be noted that when the MSE is minimized a reasonable 
definition of the signal-to-noise ratio (SNR) seen by the Viterbi 
algorithm is maximized. This is true because {q;} is considered to be 
the effective channel pulse response, constrained to unit energy; the 
additive noise plus any residual intersymbol interference is the effec- 
tive noise seen by the algorithm. Since this noise is equal to the MSE 
which has been minimized, the SNR has been maximized. 

In summary, to minimize the MSE and thus maximize the SNR 
seen by the VA receiver, choose 


Qopt = eigenvector of [J — H+A-!H] corresponding to its 

minimum eigenvalue, (9) 
Pept om AH Qont, (10) 
and then 


e? min = min eigenvalue of [J — H+A—H]. (11) 


III. AN ADAPTIVE ALGORITHM FOR OPTIMUM RECEPTION 


In order to make the receiver structure practical, the procedure of 
choosing the {p;} and the {q;} must be made adaptive since the channel 
pulse response will not usually be known prior to the start of trans- 
mission. An algorithm to choose the taps adaptively will now be 
described. 

Consider the conditions for the optimum operating point of this 
receiver to be reached. The condition that the gradient with respect 
to P be equal to zero is easily implemented by using the products of 
sampled values of quantities in the receiver as noisy estimates of the 
required cross correlations, assuming the data sequence is known or 
has been correctly estimated by the receiver. Thus, 


PH) = PO) — Ajye,RO, (12) 


where P“” is the set of tap values at the rth iteration, A; is an adjust- 
ment parameter which controls accuracy and speed of convergence, 
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R“™ is a vector of the received samples contained in the linear pre- 
processing filter, and e, is the error in (2). P‘+!) is thus the new 
estimate of the {p;} taps, and when a steady state is reached a noisy 
unbiased estimate of these taps is obtained. Note that the value of P 
implicitly depends on the value of Q through the e, terms. 

The algorithm to obtain the Q taps 1s not so easily obtained. Consider 
the unconstrained gradient with respect to the Q vector. If a noisy 
estimate of the required cross correlation is used, then the recursion 
for the unconstrained gradient algorithm is 


Q (r+1) = QQ‘) - Ave, 1™, (13) 


where I‘) is a vector of the information symbols contained in the 
channel reference filter. If (12) and (13) are followed without a con- 
straint at each iteration, then the trivial solution results. The algo- 
rithm is therefore modified so that the Q vector is renormalized at 
each step. That is, 


OCH) = OO + Age, 1”) (14) 
Q (r+1) — ee, (15) 
(O (+) + (GO (r+) 


By following the combined algorithm of (12), (14), and (15), a sta- 
tionary point in P will be reached, and the energy in Q will be con- 
strained to one. 
Now consider the noiseless unconstrained gradient of e? with respect 
to Q. Then 
0" _. 20 — 2H+P. (16) 
dQ 
Consider P to be in the neighborhood of the correct solution (6) with 
respect to Q (that is, P is adjusted more quickly than Q). Then (16) 
becomes 
Oe? 
~~ = 2Q — 2H+A“HQ- (17) 
dQ 
Thus the gradient algorithm, in terms of the actual matrix quantities, 
becomes _ 
OCH) = Q&) — ee 
dQ” 
= QO) — 1A,(20 — 2H+A7HQ“) 
= AeHtAtHQ® + Q (1 — As) (18) 


and then Q‘'+)) is renormalized to form Q‘+). Now note that if 
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As = 1 this corresponds exactly to the method of Vianello and Stodola® 
for determining the maximum eigenvalue and corresponding eigen- 
vector of Ht+A-!H. Since the maximum eigenvalue of H+A™4H cor- 
responds to the minimum eigenvalue of (I — H+A#) this technique 
will converge to the minimum MSE. This method will fail only when 
the starting vector Q™ is exactly orthogonal to the desired solution. 
Since the algorithm actually used (14)—(15) deals with noisy estimates 
rather than the exact expressions, the noise will prevent the case of 
the algorithm becoming stuck on a vector orthogonal to the solution. 

In the practical case it is not possible to choose Ae to be one because 
when the noisy estimates are used the algorithm will amplify the 
noise and diverge. Actually, A. will be much smaller than one. Again 
looking at (18), one can see that a steady state is reached when Q 
becomes nonrotating with respect to the transformation. This occurs 
when Q is the maximum eigenvalue of H+tA#H (i.e., the maximum 
eigenvalue will dominate as in the method of Vianello and Stodola). 
Thus, the unique solution for Q has been obtained. 


Iv. LIMITING RESULTS 


We now study the limiting situation where the prefilter is allowed 
to be any general linear filter with impulse response p(t), while the 
desired impulse response {qm}44, is still finite. In addition we assume 
that the additive noise on the channel is white, with double-sided 
power spectral density N ,/2. 

In this case we wish to minimize the mean square of the sampled 
error 


ay LL 
ie J p(r)r(kT — r)dr — Yo gl, (19) 
where 
(kT — 7) = ART —IT — 7), +n(kT — 7) (20) 
[=—0 


is the received signal. 
The MSE is then 


oe y a ; [ ; Dg DWIT =aphdl Sadaades 


l= — 


N, = LL 00 
9 [ p(r)*dr — 22 qi / o(r)h(UT — r)dr 


LL 
+X g@ (21) 


+- 
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Using a simple calculus of variations argument to minimize MSE 
with respect to the prefilter impulse response p(t), we get the following 
integral equation defining the optimum p(é). 


N LL 2 
= pt) = x qh(lT — t) — me SAT — 2), (22) 


where 
Se a ” p(n)h(IT — 2)dr (23) 


is the overall sampled impulse response of the channel and prefilter. 
Note that we would hope for 0 $1 S LL, Si&qi and for 1 < 0 and 
L> LL, Six 0. 

Equation (22) tells us that the optimum prefilter structure is a 
matched filter with impulse response h(—t#), followed by an infinite- 
length tapped delay line whose tap gains {pz} are given by (22) and 
(23). 


N, 
ple ae > Dibm—t (24a) 


where 


bn = h(mT — r)h(—7)dt = o_m (24b) 
is the channel’s sampled covariance function, and where we later 
require that {q:} is nonzero only for 0 $ 1 = LL. We remark that the 
development so far is analogous to that of Berger and Tufts* for the 
case LL = 0. Equation (24) may be solved in terms of z-transforms. 
Defining 


@ 


Ime” 


m=—0 


q(z) 


p(2)= SS pna™ 


o(z) = a bme™ 


we can take z-transforms of both sides of (24) and solve for p(z). 


pe) = 2), (25) 
o(z) + =y 


where we have used the fact that ¢(z) = ¢(27!) since the sequence 
{oi} is symmetric about | = o. 


ADAPTIVE CHANNEL MEMORY TRUNCATION 1551 


Using (25) we get the z-transform of the autocovarliance sequence 
of the {e,}, when the tap coefficients {p,}2. are chosen to minimize 
the mean-squared error. Defining L', = €4€44m and H(z)= > 7__ . Eme™ 
we have 

—1 
E(z) = No g(z)q(z") | (26) 


* 6@ +32 





We now minimize e? = FE, with respect to the desired impulse response 
samples {q,}Z4,, under an appropriate energy constraint. Taking the 
inverse transform of H(z) we have 


B= 2 [" wera 
Se ne te ae 


-NA i- ds, (27) 
“TIP g(e-set) 4 8 


where 
g(e#T) = qo + qe? + e+) + qrret2ZT, 


Defining the LL + 1 dimensional vector Qt = (qo, @1, -°* Gut) We 
can rewrite (27) as a quadratic form 


EL, = QTRQ, (28) 


where F# is a Square matrix of dimension (LL + 1) whose 7~jth element 
1s 





r{T ja (i—j) T 
= NT i e/ a ce (29) 


r= —__.- 
tr JWI (eset) 4 8 


Note that ¢(e%7) is the discrete Fourier transform of an autocovari- 
ance sequence, and hence is an even, real, positive function of w. Thus 
rij = 7;; is a real function of |¢ — 7|, and so # is a positive definite 
symmetric Toeplitz matrix. 

Minimization of #, under the energy constraint |Q|? = 1 is then 
accomplished by making Q that normalized eigenvector of & cor- 
responding to its minimum eigenvalue. The matrix FR is evidently the 
limiting case of the matrix J — Ht+A-!H for the finite-tap recelver 
{displayed in expressions (7) through (11) ]. Then 


Cain ae Amin (Zt). (30) 
To recapitulate, the minimum is taken over the set of tap-coefficients 


{pn}e.—«. and {g,}Z4, under the constraint 44, g = 1, 
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which can be expressed as 

Pf gerd = 1 
Thus, from (27) we have the lower bound 


Amin(R) = By = Gin 2 oa 
sup (oe#") +5) 
—1r/T Sw S2r/T 2 


Now ¢(e%7) is the discrete Fourier transform of the sequence {¢n} 
defined in terms of the channel’s impulse response by (24b). Thus, if 
the channel’s transfer function is denoted by 


(31) 


Ho) = [ ” h(t)e-#T db , 


H (w+ “a ) | (32) 


The term ¢(e%*7) can be interpreted as the channel’s ‘‘folded”’ power 
spectrum.’ 

When LL + 1, the number of components in the desired impulse 
response {q,}Z4,, is relatively small, say less than 10, the minimum 
eigenvalue and corresponding eigenvector of FR can be evaluated with- 
out difficulty. For much longer values of LL, the lower bound (81) 
which is easily computed using (82) may be quite tight. A particular 
case of interest is where LL = 1. Then R has the form 


R on To T1 
T1 To 


sem =a ¥ 


rn=—o 











and 
€min = Amin(R) = min(ro + 11, To — 11), 
where 
NT pelt cos? 
retin = / oor (33) 
—7/ (e%#T) + “3 
and 
NT pelt sin’ 2 
ron = =e | isi (34) 


TIT 4 (giv?) 4 af 
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The normalized eigenvectors (optimum (q,,g1)) corresponding to the 
eigenvalues ro +71 and r, — 71 are respectively (1/v2, 1/v2) and 
(1/v2, —1/v2). 

Thus, we have the curious result that the optimum desired impulse 
response of length two is one of only two possible forms, depending 
only on whether the channel’s folded power spectrum is such that 
(33) or (34) is smaller. For example, if the channel’s folded power 
spectrum has a single minimum near the band edge, w = 7/T, the 
best choice for (q.,g1) would be (1/v2, 1/v2) since cos? (w7')/2 has a 
zero at the band edge. However, if the channel’s folded power spectrum 
has a single minimum near zero frequency, the best choice for (qo,q1) 
would be (1/v2, —1/v2), since sin? (w7')/2 is zero at w = o. These 
two cases are illustrated in Fig. 4. 

It is interesting to point out that the two possible optimum desired 
impulse responses (1/v2, 1/v2) and (1/v2, —1/v2) are reminiscent 
of duobinary and partial response impulse responses.® 


V. PERFORMANCE OVER SIMULATED CHANNELS 


In order to observe performance obtainable from this receiver struc- 
ture, the arbitrary discrete time channels shown in Fig. 5 were used. 
Figure 6 shows the results of the simulations performed with the 
receiver developed here and that of Qureshi and Newhall on these 
channels. Underlined in Fig. 5 are the desired response used for the 


CHANNEL’S FOLDED OPTIMUM DIR POWER SPECTRUM 
POWER SPECTRUM 


| Gotq,elot |? 





[q,, 94] , 
ils _- | | 
CASE 1 v2 v2 | | 
0 T | | 
| | 
“7 0 a 
T T 
| ne 
v2 | | 
| | | 
| ! | 
CASE 2 | ; = | | 
| | | 
bald 0 = “t- Go : = 
T i 2 i i 


Fig. 4—Optimum desired impulse response of length two. 
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~0.1, -0.2,-0.1, 0.3, -0.3, 0.33, 0.67, 1.0, 0.67, 0.33, 0.3,-0.5, -0.1, 0.1, 0.1 


CHANNEL A 


-0.1, -0.1, -0.3, -0.15, -0.33, 0.16, 0.33, 1.0, 0.33, 0.16, -0.33, -0.3, -0.15, -0.1, -0.1 


CHANNEL B 


Fig. 5—Sampled channel impulse responses used in the simulation. 


Qureshi and Newhall receiver. As can be seen from the performance 
curves, the Qureshi and Newhall receiver performs about as well as 
our receiver for Channel A and much worse for Channel B. The 
difference in performance is presumably due to the use of different 
criteria to choose the desired response. In the case of Channel B, the 
channel passes virtually no dc, yet the DIR from truncating the 
channel response does pass de. This causes considerable noise enhance- 
ment by the Qureshi and Newhall linear prefilter. 

Figure 6 also shows the matched filter lower bound, the lower bound 
on performance derived by Forney,® and a lower estimate which is 
used to predict actual optimum reference receiver performance. This 
lower estimate is obtained by computing the MSE and minimum 
coding distance of the DIR. Thus, it is assumed that the MSE 1s un- 
correlated and Gaussian in this approximation. 


re 1 [| @, 
- | a 5 
P(e) SK erte (4 sia) (35) 


where K is a constant depending on the error structure of the channel, 
and dmin is the minimum Euclidian distance between all possible 
pairs of noiseless sequences with differing first information symbols 
emerging from the prefilter.? This lower estimate is found without 
considering the fact that the noise is correlated. If a more accurate 
estimate of performance is desired, the results of Qureshi and Newhall‘ 
can be used to consider the effects of noise correlation. 

The simulations were run with a 31-tap prefilter whose taps were 
adjusted with A = 0.001, and a 5-tap desired overall response length 
with the adjustment parameter equal to 0.01. In the case of the Qureshi 
and Newhall receiver, the prefilter was adjusted with A = 0.001 and 
the channel was estimated with a filter with adjustment parameter 
equal to 0.01.* As the curves show, the receiver structure given here 


*The performance loss due to the adjustment parameters has not been evaluated ; 
however, simulation results indicate that this loss is very small. 


p(e) 
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Fig. 6—Simulation results: (a) channel A; (b) channel B. 


assures that a good choice of a DIR is made which is not always the 


same as the truncated channel impulse response. 


VI. COMPARISON WITH OTHER SYSTEMS—AN EXAMPLE 


Based on the results in Section II, performance calculations were 
made for baseband PAM transmission on the channel whose frequency 
response is shown in Fig. 7. The results shown in Fig. 8 were made with 


the following assumptions: 


(z) A matched filter preceded the receiver. 
(22) There was a 31-tap prefilter. 


(420) There was a 5-tap desired impulse response. 
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Fig. 7—Channel amplitude characteristic. 


(2v) Although the noise may be correlated, at the input to the 
Viterbi algorithm, it has a negligible effect on performance. 


Of the assumptions made, only the one about the noise correlation 
might not be realistic. Work is currently being done to deal with the 
correlated noise problem. In any case, it is not expected that it would 
affect performance more than a few dB and it clearly would not affect 
the place in the performance curve at which the performance begins 
to degrade seriously. 

The curves representing the linear and decision-feedback equalizers, 
provided by J. Salz, show the MSE versus rate for additive white 
Gaussian noise with N,/2 = 0.0001. In the linear and decision-feed- 
back cases the MSE may be roughly related to performance in terms 
of probability of error.**®!° The curve for the prefilter/VA combination, 
labeled ‘““VA equalizer,” is a plot of (MSE/d?2,,) versus rate, where 
dmin 18 the minimum distance for the DIR. This is done because the 
attainable system performance is not determined by MSE alone, but 
rather by MSE/d?,,, as in expression (35). Direct minimization of this 
ratio by analytical or numerical means has not been accomplished. 
Note however that the minimum value dyin can attain (over all 


“The analysis for decision-feedback equalization ignores the effect of decision 
errors on the MSE. 
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Fig. 8—Indication of attainable performance for three receiver structures. 


channels with equal-energy impulse responses) is limited by the dura- 
tion of the DIR which is chosen." 

As can be seen from the curves, this receiver structure can be ex- 
pected to perform well while using only binary signaling over a much 
greater range of transmitted data rates than the linear and decision- 
feedback receivers. This result occurred despite the fact that the 
linear and decision-feedback computations were made for infinite 
filters while the prefilter was finite. It is the more relaxed criterion for 
our system compared to the decision-feedback criterion which results 
in lower MSE and thus better performance. Nevertheless the results 
are considered preliminary until a better understanding of the effect 
of noise correlation is achieved. 


VII. TRANSMITTER OPTIMIZATION 


The ‘‘channel’s’”’ frequency response H(w) actually includes the 
transmitting filter, 1.e., 


H(w) = C()G(), (36) 


where C(w) is the frequency response of the transmission channel 
alone, and G(w) is the frequency response of the transmitting filter, 
which we have hitherto assumed fixed. In a practical data communi- 
cation system, a ‘reasonable’ transmitting filter would likely be 
fixed to avoid having to provide an extra feedback channel for adjust- 
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ing the transmitter parameters, and because of the complexity of the 
transmitter optimization argument itself for general channels.® 

Nevertheless, the performance attainable with transmitter optimi- 
zation is of theoretical interest. In this section we obtain expressions 
for the optimum transmitter filter G(w) and the resulting minimum 
MSE under a transmitted power constraint. For simplicity, we assume 
a “well behaved” channel C(w) for which | C(w)| is monotone decreas- 
ing, and for which |C(w)|/N. is sufficiently large in the range { —7/T, 
a/T} that the optimum transmitter uses the entire Nyquist band 
|w| <= +/T. Treatment of more general channel characteristics is more 
complicated, but can be carried out as in Ref. 6. 

The minimum MSE for a fixed transmitting filter G(w) and DIR {q:} 
is given by eq. (27) and by the channel’s folded power spectrum, 
which from (32) and (36) can be written 

arn 
G(» ~ =F") 


sem) = 7 Y |C(o- Fr) 


The constraint that the transmitted power be fixed at Pr can be 


written 
o(0 38) 


A necessary condition for minimizing the MSE, given by (27), 
subject to the power constraint (38), is obtained using a simple vari- 
ational argument: for —7/T S w S 7/T and every integer m, either 
G(w — 2xrm/T) = 0 or G(w — 2rm/T) ¥ 0 and 


I < 2rn \ |? 2rn\ |\2. N, 
7. [¢(e-F)| [@(e-4r) | +3 


=rJa(e#")| | C(o — H" ) 


where J is a Lagrange multiplier whose value will be determined from 
the constraint (88). Furthermore, for any w such that C(w) = 0, 
G(w) = 0. 

For any frequency w, there will be only one integer m for which 
G(w — 2rm/T) # 0, since the left-hand side of (39) does not depend 
on m and the right-hand side does. Indeed, if |C(w)| is monotone 
decreasing, then best use is made of the transmitter power if G(w) = 0. 
for |w| > 7/7. Thus we can rewrite (39) as 


2 2 











(37) 





1 a{T 
TF a 


—r/T n 


2 
dw = Pr: (38) 




















,» (39) 





al C(a)|* |G(w) |? +S? = rlale#”)| [C()| 
for C(w) ~ Oand |w| S w- (40) 
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For simplicity, we assume that Pr and the ratio |C(w)|/N. are 
sufficiently large that (40) can be satisfied for all |w| < 7/7. [Other- 
wise G(w) would be zero® beyond a certain frequency wo < 7/T. | 
Then the amplitude frequency response of the optimum transmitting 
filter is given by 


I » .. Alg(e?*)| No us 
fea, sce lag 
—() for w > * (41) 


The Lagrange multiplier \ is determined by (41) and the power 
constraint (38). Substitution of the expression for |Gopt| into expres- 
sion (27) for the MSE gives 


Nol (7! \ale**)| 





= Jd\e I g 
MSE Ea Jee (CG) w , (41a) 
where 
r/T 
Prt aT 
sei (41b) 


ce a g(e*?)| : 
2/7 |C(w)| 


It is interesting to look now at the frequency response of the optimum 
receiver prefilter 


1 [Pa(o)l? = [Clo)? |Gone(w) |? [pe*™12, (42) 


corresponding to the optimum transmitter filter. The left side of (42) 
follows from the cascade of the channel and transmitter and the appro- 
priate matched filter, followed by the discrete filter {p:}. Substitution 
of expressions (41) for |Gopt(w)| and (25) for p(e#7) results in 


J _ 1 [ Alg(e#?)| No 
plPa)!= 0] Yea acme] fr lel § 
Thus the transmitting filter and receiving prefilter frequency re- 
sponses are identical in the Nyquist band except for constant factors 
(clearly, the transmitting and receiving filters’ phase characteristics 
can be chosen arbitrarily). This equal sharing of the filtering load 
between the transmitter and receiver is a well-known result for 
optimum linear communication systems (see pp. 118-121 of Ref. 7). 
It is also of interest to evaluate the power spectrum of the error 
sequence that the Viterbi algorithm assumes to be additive uncorre- 


7 (48) 
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lated Gaussian noise samples. From expressions (26) and (40), this is 
given by 


No __|q(e%?)|? 





2 Al g(e*7)[[C()| 
_ N.|g(e%") 7 


Thus, the extent that the amplitude frequency response of the 
chosen DIR approximates that of the channel in the Nyquist band 
determines how close the power spectrum H(e#") is to being flat, 
and hence, to what extent successive errors are uncorrelated. 

From (41a) and (41b) we obtain an expression for the MSE for a 
given DIR after the transmitting and receiving filters have been 
optimized. 


N oT 
“= a(Q) 
eos Or aC) a 
where 


TI? | g(e%?)| 
= foe oT 


LL LL i 
TL & wnrmn 
i Snead (117) aoe (45b) 


Minimization of the MSE expression with respect to the DIR Q 
under the constraint |Q|? = 1 is then equivalent to minimization of 
a(Q) under this constraint. Necessary conditions for the optimum 
Qt = (qo, -++ dzz) are then 


LL 
kqQi = 2X, Qmpi—m(Q), l= 0, Sa LL, (46a) 
where 
r/T eilol 
iO) / dis (46b) 


Saas EE LL eel 
io()i] 2 gat" | 


and where uw is a Lagrange multiplier. Again the optimum DIR Q 
is the solution of an eigenvalue problem, this time nonlinear. It is easy 
to verify that the optimum DIR of length two is again either of the 
two “duobinary” impulse responses shown in Fig. 4. In that case the 
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minimum achievable MSE is given by (45a) and (45b) as 


NLT pel? [1 & cosoT} , 7 
Ar if T | C'(w) | dew 

MSE = a ee pe (47) 

Pp HDs tph 

PT 2 Jia [C(@) 2 


the + or — being chosen to minimize (47). 





VIII. CONCLUSIONS 


We have presented a scheme of linear prefiltering to optimally 
“condition” the impulse response of a channel to approximate an 
impulse response of limited duration for which maximum likelihood 
estimation of the data sequence is implementable in practice. This 
scheme in conjunction with the VA can be adaptive, to deal with un- 
known or slowly time-varying channels. In the simulations its per- 
formance compared favorably with the similarly motivated scheme of 
Ref. 4. 

The optimization criterion we used—minimization of the MSE with 
respect to the prefilter taps and the DIR, with the energy, duration, 
and relative delay of the DIR being fixed—is admittedly somewhat 
ad hoc. If the sequence of errors {e;) emerging from the prefilter is still 
assumed to be stationary Gaussian, with zero mean and covariance 
{Em}, then it can be shown that the error rate of a VA which assumes 
correlated noise is minimized if a certain weighted minimum distance 
iS maximized, namely 

min d+A—'d, 

dGs’ 
where the s’ is the set of all possible vectors representing error events 
and A is a covariance matrix whose dimension equals that of 
d{A,; = E\,_;}. The above quantity is clearly difficult to maximize, 
and even if it could be done, the non-Gaussianness of the error se- 
quence would render the solution suspect. 

Nevertheless, the performance estimates for the sample channel 
reported in Section VI make the use of the VA is conjunction with 
prefiltering appear attractive for high-speed data transmission relative 
to other schemes. Further studies should be done on the correlatedness 
of the error sequence and the minimum distance properties of the 
desired impulse responses. * 


*S. Fredricsson presented a paper dealing with this subject at the International 
Symposium on Information Theory, Israel, June 1978. 
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Multimode Theory of Graded-Core Fibers 


By D. GLOGE and E. A. J. MARCATILI 
(Manuscript received March 29, 1973) 


New technologies of fiber manufacture and a demand for unusual fiber 
qualities in communication systems have intensified the interest in a 
comprehensive theory of multimode fibers with nonuniform index distribu- 
tions. This paper deals with a general class of circular symmetric profiles 
which comprise the parabolic distribution and the abrupt core-cladding 
index step as special cases. We obtain general results of useful sumplicity 
for the impulse response, the mode volume, and the near- and far-field 
power distributions. We suggest a modified parabolic distribution for best 
equalization of mode delay differences. The effective width of the resulting 
impulse is more than four times smaller than that produced by the parabolic 
profile. Of course, practical manufacturing tolerances are likely to in- 
fluence this distribution. A relation is derived between the maximum index 
error and the impulse response. 


I. INTRODUCTION 


Conventional optical fibers consist of a high-index core surrounded 
by a cladding of lower index. The index step between core and cladding 
contains the light inside the core and isolates it from the outer fiber 
surface, whose quality is usually difficult to control. In a more general 
way, inside guidance can be accomplished by any index profile which 
decreases from a maximum inside the fiber to a lower (cladding) value. 
The specific shape of the profile has an effect on the distribution of the 
guided optical power in the fiber and on the overall loss encountered, 
but, more importantly, the profile profoundly influences the velocities 
of the various propagating modes. A good example is the parabolic 
index distribution which was predicted to nearly equalize the group 
velocities of the propagating modes.! The Selfoc fiber which closely 
approximates these conditions has indeed since exhibited an extremely 
narrow impulse response.°4 

These effects greatly enhance the chances of multimode fibers to be 
used in optical communication systems. On the other hand, a theory 
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of the interrelations between index profile, impulse response, and 
power distribution is presently only available for the two special cases 
of the uniform and the parabolic core index. This paper provides a 
more general theory and studies a broad class of index profiles po- 
tentially useful in communication applications. The uniform and the 
parabolic profile are special cases within this class. 

Our concern with multimode fibers for communication applications 
allows us to make four simplifying assumptions: 


(2) The index profile is circular symmetric. 
(72) The core diameter measures hundred wavelengths or more and, 
hence, a great number of modes can propagate. 

(iz) The total index change within the guiding core region is only 
a few hundredths, so the propagating modes can be considered 
essentially as transverse electromagnetic.® 

(tv) Index variations within the distance of a wavelength are 
negligible, and the conditions of geometrical optics (or the 
zeroth order of the WKB method) apply. 


Except for these four restrictions and the requirement of guidance, 
the index profile can be of the most general form. It can, for example, 
have an index depression in the center and one or several ring-shaped 
index maxima.® 

For the sake of clarity, this paper is restricted to the simpler type 
of profile illustrated in Fig. 1. We assume the index profile will decrease 
monotonically from the center and converge into a flat cladding region 
which guarantees isolation from the outside surface. 


INDEX ———> 


CLADDING CLADDING 





RADIUS ———-> 


Fig. 1—Cross-sectional sketch of circular symmetric index profile in multimode 
fiber. 
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Fig. 2—Wave vector diagram in the propagating region of a multimode fiber. 


Apart from the index profile, there are, of course, other influences 
which affect the impulse response and the optical power distribution 
inside the fiber. Mode excitation, loss differences in the process of 
propagation, and coupling among the modes play a part. To isolate 
the effect of the index profile, we assume here the ideal case of uniform 
loss, absence of coupling, and equal and simultaneous excitation of all 
propagating modes at the input. For the computation of the impulse 
response, the input is assumed to be an infinitely narrow pulse of unit 
energy. 


II. MODE DESIGNATION AND MODE COUNT 


All guided modes are essentially transverse electromagnetic and, 
with some proviso, can be decomposed into linearly polarized pairs.*: 
Because of the circular symmetry of the index n, the modes have a 
circular periodicity and can be identified in the conventional way by 
an azimuthal order number v. To characterize the radial field distribu- 
tion, we need an additional mode number yz. The propagation constant 
8 of a particular mode (yp, v) can then be approximately determined 
by the WKB method.®® Figures 2 and 3 give a physical description 
of these relationships. In Fig. 2, the local wave number 


k(r) = 2rn(r)/r (1) 
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Vy «2 (r)- 8? 


x 


ae 


Fig. 3—Sketch defining regions of periodic and aperiodic field characteristics of a 
mode of azimuthal order »v. 


is decomposed into its components in a cylindrical coordinate system 
(r, ¢, 2). The unknown radial component becomes 


u(r) = [ke (r) — B® — 2/7? Hf. (2) 


Given 6 and », we can find two radii R; and Re, at which u(r) vanishes 
(see Fig. 3). These radii define a ring-shaped region within which w is 
real, causing a radial periodicity of the mode field. Outside this region, 
the field is aperiodic. 

Radially decreasing (or evanescent) field conditions obtain outside, 
when the phase inside (approximately) adds up to an integer number 
of half periods between R; and Re. Consequently, if u designates this 
number of half periods, 


= J : Ue / 7 [k(r) — ? — v?/r? dr. (3) 


We would have obtained the same result by way of the WKB method, 
with the only difference that » and v? would be replaced by » + 4 and 
v2 + 4. These corrections are important in the case of small yu or », 
and particularly for the fundamental mode which has p = v = 0. On 
the other hand, to obtain a general view of the mode structure, we 
can ignore the 3-terms as long as we refrain from discussing individual 
low-order modes. 

For the purpose of a total mode count, let us consider the limits of 
nu, v, and 6. The requirement of evanescent field conditions in the 
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cladding (index n, in Fig. 1) limit 6 to a minimum value 
Be = 2Qan,/d. (4) 


Modes with smaller 8 find propagating conditions in the cladding and 
are no longer bounded by the core profile. Condition (4) defines mode 
cutoff. The largest value for v results for B = 6. and » = 0, and 
alternatively u is largest for 8 = B. and v = 0. We obtain the total 
number of modes M from a summation of (3) over all v from 0 to 
Vmax. Lf Ymax 18 a large number, we may consider v a continuous variable 
and replace the sum by an integral. In this case, 


4 Vmax Rev) é 
wae i] i [h2(r) — 62 — v2/r Pdrdy. (5) 
T J0 Ry”) 

The factor 4 in front of the expression allows for the fact that each 
combination yp, v designates a (degenerate) group of four modes of 
different polarization or orientation.’ Figure 4 illustrates the area of 
the double integration indicated in (5). A change of order in the 
integration leads to 


4 fe prck—p2)? 
a J / (k2 — B2 — /r)tdydr , (6) 
0 0 


where a is the radius at which the index n(r) reaches the cladding 
value n,. Integrating (6) with respect to v yields 


M = f° De@) — Beprar = (FE) flee) — Prd. 


For small index differences, the integral represents the volume under 
the (circular symmetric) profile plot. It may be worth noting, though, 
that the substance of this relation 1s not limited to circular symmetry. 


Vv 


0 R 1 (v} R2(v) 
RADIUS —> 


Fig. 4—Region of double integration in eq. 5. 
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INDEX n —— 





Ne 


RADIUS r ——pe 


Fig. 5—A few of the index profiles defined by n = noL1 — 2A(r/a)*]? for small A. 


For later use we write (7) in the somewhat different form 


R2(0) 
m(6) = fo [h(r) — 6*Prar, (8) 


where m(8) denotes the number of modes having a propagation con- 
stant larger than 8. The upper limit &2(0) of the integration is the 
radius at which k(r) = 8B. 

Let us now consider a particular class of profiles defined by 


Noll — 2A(r/a)* |} for r<a 
n(r) = (9) 
Nol1 — 2A}? for r>a, 


where a is a parameter between 1 and ©. Figure 5 illustrates the cases 
a = 1, 2,4, 10, and . All profiles reach a constant cladding value at 
r = a. The core profile has a cone shape for a = 1, becomes nearly 
parabolic for a = 2, and converges to the case of the step profile for 
a = o, Using (1) we introduce (9) into (8) and obtain 


= ; a k? 29 B? (2/a)+1 
m(8) = a?Ak? onenag (“Se ) (10) 
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where 
k, = 2rn,/X. (11) 


For 8 = @. from (4), the total mode number becomes 





M = ak . (12) 


a 
a+ 2 
It is proportional to the index difference and the core cross section. 
The uniform profile accepts twice as many modes as the parabolic one 
and three times more than the cone-shaped one. 


III. IMPULSE RESPONSE 


Consider all modes to be excited by the same narrow pulse at the 
input. Each mode transports an equal amount of energy to the fiber 
end. The individual pulses are expected to suffer a certain distortion, 
depending on the 6-w characteristic of each mode and dispersion in the 
dielectric. We assume, however, that the resultant broadening is small, 
or at least not much larger than the group delay differences between 
adjacent modes. Because of this effect and other limitations in the 
system response, the pulses from individual modes are likely to fuse 
into one continuous output pulse called the impulse response. Since 
all modes carry the same energy, the power profile of the impulse 
response is equal to the mode density per unit time interval. In the 
following theory, the continuity of the impulse response results not 
from the broadening of the individual mode responses, but from the 
assumption that » and »y are continuous functions. 

The straightforward method of computing the impulse response 
starts from (3) to find the propagation constant 8 for each pair, y, ». 
The group delay in a fiber of length L is then 


r(u, ») = ae Bl (13) 


where c is the vacuum velocity of light. A simplification of this ap- 
proach for the purpose of numerical computations is indicated in the 
appendix. Once (x, v) is known, the impulse response results from 
a count of the combinations yu, v which arrive between 7 and 7 + dr. 
This number plotted versus 7 then constitutes the impulse response. 
For the special class of profiles defined by (9), group delay and 
impulse response can be computed in a much simpler way. First we 
postulate that, in this case, the relation between 7 and @ according to 
(13) is independent of » and »v. If this holds—and we shall prove it 


1570 THE BELL SYSTEM TECHNICAL JOURNAL, NOVEMBER 1973 


later with the help of eq. (16)—we can replace 8 by 7 in (3) and still 
perform the same integration over v which led to (8) and, more specifi- 
cally, to (10). Solving the result of this integration for 7 yields 


Lite d-\ 55 2ma+ 2 
ap |e 


c ak, 


Tt = 








a | (a-+2) } 
) (Banger | (4) 


This result can easily be verified by solving (10) for 8 and introducing 
it into (13). With the help of (10) and the abbreviation 


6 = 3(1 — 6?/kz), (15) 
eq. (14) takes the form 


_ In.1 — 48/(@+ 2) 
ar: (1 — 26) 





T (16) 
This expression proves indeed to depend on @ alone (and not explicitly 
on m), thus justifying the approach chosen. 

To obtain the impulse response, we can now introduce (16) into 
(10) and differentiate with respect to 7. Although this is not difficult 
to do, it leads to rather unwieldy expressions. We shall therefore merely 
consider some special cases of interest. To normalize the impulse 
response for total unit energy, we divide (10) by (12) and obtain 


§ \ @le)+1 
a7(x) an) 


Furthermore, since 6 can at most assume the value A (for 8 = 8.) 
and is therefore small compared to unity within the scope of our theory, 
we develop (16) into a power series in terms of 6 and obtain 


Ln, a—2 3a — 2 & | 
rap (itigg'+ Seas) as 





We relate 7 to the total propagation time Ln,/c and introduce a new 
time reference, which ignores the delay common to all modes. Hence, 


TC a— 2 3a — 20° 
7 a+2 2 
In this time frame, the fundamental mode arrives at ¢ = 0. 


As long as a is not too close to 2, the linear term in (19) dominates. 
Therefore, 


(19) 


a+ 2 


t except for aW2 
oe eel (20) 


2: for a= 2. 
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Insert this into (17) and differentiate with respect to ¢ to obtain the 
impulse response 

















(2/a)+1 
sel SR |¢| 2/4 except for a2 
a a—2A 
1 dm 
2 f _ 
Az or a= 2. 
As 6 varies from 0 to A, the time ¢ changes from 0 to 
a—2 
f ows) 
aS, A except for aW2 
{ae (22) 
A? 
3 for a = 2. 


Outside of this time interval, the impulse response is zero. Figure 6 
shows plots of (21) for the profiles sketched in Fig. 5. A change from 
a= o to a = 10, which implies a relatively small change in the 
profile, narrows the impulse response by 3. The response becomes 


oh 


10 
A 


[>|c0 


D>|o 


bls 


|r 





-1/3A 0 1/3A 2/3A A 
RELATIVE DELAY, t 


Fig. 6—Impulse response of multimode fibers having the profiles of Fig. 5. 
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extremely narrow for a2, then broadens again, as a decreases 
further. For a < 2, the high-order modes overtake the fundamental 
and arrive earlier. 

In the vicinity of a = 2, where both terms of (19) contribute, the 
impulse response is a rather complicated function. The most interest- 
ing of these cases is the one for which the impulse response has the 
narrowest possible width. This optimum condition arises for 


Qopt = 2 — 2A, (23) 
which yields 


t = 3(82 — Ad). (24) 


In this case, the modes of highest and lowest order both arrive at the 
same time ¢ = 0; all other modes are faster, the fastest one being 
determined by 6 = A/2. It arrives at 


2 
t= — 5. (25) 


Equation (24) has two solutions for 6. Hence, (17) yields two values 
for the same #, indicating that two mode groups, a high and a lower 
order, contribute to the impulse response at every particular instant 
in time. By introducing 6 into (17), differentiating with respect to t, 
and then adding the two contributions, we find the impulse response 


4 1\-} 
a(it+z) | (26) 


This function is plotted in Fig. 7. It peaks até = — A?/8 and decreases 
towards t = 0. Because of the normalization introduced in (19), the 
absolute temporal width is 





Ln, De. 


c 8 a7) 


The time slot in which a pulse of this kind can be transmitted is 
narrower than that, because 70 percent of the power is concentrated 
in the first half of the interval (27). 

A practical implementation must, of course, allow for a certain 
tolerance or error in the profile, as a result of which the total width 
of the impulse response is likely to exceed (27). To obtain some in- 
dication of the pulse broadening as a result of this index deviation, we 
assume that the erroneous profile is still of the type (9), but has 


Qa = Aopt + da. (28) 
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-0.15A2 -0.1A2 -0.05A2 0 
TIME 


Fig. 7—Impulse response in the case of optimal profile shape. 


The maximum index deviation from the optimum profile then appears 
approximately at 

r = ae? (29) 
and has the value 
NA 


Anmax = da ae? 





(30) 


where e is the base of the natural logarithm. As a result of this profile 
error, the normalized width of the impulse response becomes 


g(A + 3|da|)? (31) 
or, in absolute terms, 


In, € 2 
Gn (2 + a Janes! (32) 


Consider a guide with a maximum index n, = 1.5 and an index 
variation A = 2 percent. If the profile is optimal, mode delay should 
produce an effective broadening of only 0.25 ns/km. An index devia- 
tion of 10-4 from the optimal profile increases the broadening to 
0.53 ns/km. 





IV. NEAR- AND FAR-FIELD POWER DISTRIBUTION 


We take again into account the fact that the core cross section mea- 
sures many wavelengths in diameter. If this cross section is illuminated 
by an incoherent source (exciting all modes uniformly), the power 
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incident per unit solid angle at any point in the cross section is con- 
stant. To compute the power accepted by the fiber, we merely have to 
know the solid angle of acceptance at any point. We find this angle 
from the wave vector diagram of Fig. 2, which yields 


cos O(r) = B/k(r). (33) 
The maximum angle 6, results for 8 = B,; hence, 
— Pe _ _te | 
cos 6.(r) = ca ae) (34) 


Using this relation, we can define a local numerical aperture at the fiber 
front face 
A(r) = n(r) sin 6.(r) = [n?(r) — n?}}. (35) 
The power accepted at 7 is then 
= a m(r) — 1; 
p(r) = p(0) A2(0) ~ p(0) (0) = 7 (36) 


If all modes propagate equally attenuated and without coupling, the 
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Fig. 8—Power distribution in the core of multimode fibers having the profiles of 
Fig. 5. 
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ae 9—Power distribution in the far field of multimode fibers having the profiles 
of Fig. 5. 


same power distribution should hold for the fiber end face. The class 
of profiles described by (9) has 


A(r) = no(24)*L1 — (r/a)*}* (37) 


p(r) = p(O)[1 — (r/a)*]. (38) 
The agreement between the profile plots (Fig. 5) and the near-field 
power plots (Fig. 8) is not a coincidence, but holds in general as long 
as the total index variation is small. 

Under the conditions assumed here, every incremental area of the 
core cross section at the fiber end uniformly illuminates its cone of 
acceptance. For this reason, all those areas that have a numerical 
aperture 


and 


A(r) = sin 0 (39) 


contribute equally to the far-field power at 0. For the class of profiles 
described by (9), the areas contributing to @ are within a circle whose 
radius is obtained by solving (37) for r. Consequently, 


P(6) = P(0) (1 = an) (40) 
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is the far-field power distribution. Figure 9 shows a plot of (40) for 
the profiles of Fig. 5. The uniform illumination for a = © changes 
to a parabolic distribution for a = 2. All plots must be understood as 
the average power expected under the idealized conditions mentioned 
earlier. Monochromatic mode excitation results in mode interference 
phenomena and a local fine structure, which can greatly modify the 
average distribution considered here. 


V. CONCLUSIONS 


By assuming somewhat idealized conditions for mode excitation, 
coupling, and loss in a multimode fiber, we can isolate the influence 
of the index profile upon mode volume, near- and far-field power dis- 
tribution, group delay, and impulse response. Surprisingly simple 
relations exist for a special class of profiles which comprises most 
multimode fibers of interest. Particular attention is given to a near- 
parabolic profile which accomplishes optimal delay equalization of all 
modes. If the (relative) index difference between center and periphery 
of this profile is A, mode delay broadens the impulse response by a 
fraction A?/8 of the total propagation time. This amounts to about 
0.25 ns/km for A = 2 percent. On the other hand, an index deviation of 
10-4 from the optimal profile increases the broadening to 0.53 ns/km. 


APPENDIX 
Some Further Relations for the Group Velocity 


The numerical evaluation of 8 as a function of u, », and k, and its 
subsequent differentiation to obtain 7 are usually tedious and time- 
consuming. A substantial simplification results from a direct computa- 
tion of + by applying the operation 





_ Ln, Ou/ Ok» 
~  ¢ dp/dB (41) 
to (38). The result is 
Re 
L J k(r)n(r)dr/u(r) 
oe eens (42) 


| ” Bdr/ u(r) 
Ry 


To understand the physical significance of this relation, consider a 
ray propagating along the fiber core of Fig. 2 in such a way that it has 
the direction of k(r) at r. A line element along this ray is 


ds = (dr? + r°dg? + dz?) (43) 
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and therefore 


ds KO) ng 
dr 


ace BL 
dr = u(r) 


u(r) 


The condition u(r) = dr = 0 at R; and R, indicates a reflection (turn- 
around) of the ray. The ray performs periodic undulations between 
Ri and Re, simultaneously moving sideways in a helical fashion. By 
introducing (44) into (42), we obtain 


Sf mas) 
p ae 


where ¢ denotes integration over a full period of the ray. The de- 
nominator describes the axial length of one ray period, and the nu- 
merator the propagation time along the ray within this length. 
Multiplied by the fiber length, this ratio yields the total group delay. 
This result emphasizes the equivalence between ray theory and the 
zeroth-order WKB approach followed in this paper. 

Within this order of approximation, the only quantities that depend 
on the wavelength are the mode numbers. Normalization of these 
numbers and subsequent transition to continuous variables eliminates 
the wavelength entirely; group velocity and impulse response are then 
independent of wavelength. More specifically, if we write 


(44) 


T= 


(45) 


p = p/ak, and o = v/ak, (46) 
and 


n= no[l — 2d(r)]}', (47) 


eq. (3) assumes the form 
1 
pees J [23 — 2d — (oa/r)* Par, (48) 
TA JR, 
and (42) becomes 
Re 
/ (1 — 2d)dr/[28 — 2d ~ (ca/r)?} 


$5 Ln, Ry, 
a R 
i * dr/[28 — 2d — (ca/r)?}} 
Ry 


7. (1 — 26)-3 


(49) 


These two equations are sufficient to calculate group velocity and 
impulse response in the case of large mode numbers. 
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Optical Fiber End Preparation for 
Low-Loss Splices 


By D. GLOGE, P. W. SMITH, D. L. BISBEE, 
and E. L. CHINNOCK 


(Manuscript received May 8, 1973) 


Cables made from brittle materials like glass require new techniques 
of end preparation for the purpose of splicing, especially if such splices 
are to be made in the field. We report here on a method of breaking fibers 
ina way which invariably produces flat and perpendicular end faces. 
We explain the underlying theory and derive optumal parameters that 
permit the design of a simple breaking tool. Experiments with a tool of 
this kind show that the tolerances for successful fracture are not critical. 
Laboratory splices of multimode fibers prepared by this method exhibited 
losses of less than 1 percent (0.04 dB) when joined in index-matching flucd. 


I. INTRODUCTION 


With installation and maintenance consuming an ever-larger share 
of system costs, simple and inexpensive splicing techniques have be- 
come a prerequisite for competitive communication systems. One 
bottleneck in optical fiber cable splicing is the fiber end preparation, 
as conventional grinding and polishing techniques turn out to be time- 
consuming and costly, especially in the field. It is well known that 
glass fibers sometimes break with flat and perpendicular end faces if 
they are previously scored,! and it has thus become common practice 
in the laboratory to obtain good ends in this way by trial and error. 
Besides being faster and simpler, this technique has the added ad- 
vantage of producing perfectly clean surfaces uncontaminated by 
lossy residues. Such ends were recently used in fiber joining experi- 
ments to determine eventual splice losses.?—> The lowest losses obtained 
were about 10 percent for single-mode fibers*:® and 3 percent for multi- 
mode fibers.? 

For such laboratory practice to become useful technology, absolute 
control of the breaking process and utmost reliability in obtaining a 
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successful result are required. We report here on an approach which 
guarantees this reliability through control of the stress distribution in 
the fracture zone. The break is initiated by lightly scoring the fiber 
periphery at the correct point. We explain the underlying theory which 
allows us to predict the character of the break from the initial stress 
distribution. By modifying a previous design,® we obtained a simple 
tool that permits us to vary the amount and distribution of stress in 
the fracture zone. All 180 breaks we have made with this tool have 
produced the predicted fracture surface. The range within which per- 
fectly flat and perpendicular end faces were obtained was found to be 
so wide that the eventual construction of a simple hand tool for this 
purpose should present no problem. The quality of the surfaces ob- 
tained makes this method the most promising of all the techniques 
investigated so far.’~® This notion is supported by some fiber-joining 
experiments which we describe in Section IV of this paper. Low-loss 
multimode silica glass fibers were prepared by our breaking technique 
and then joined in an index-matching liquid. With proper alignment, 
the splice losses were always less than 1 percent. Results on alignment 
tolerances for multimode fiber splices are also given in Section IV. 


II. BRITTLE FRACTURE OF GLASS RODS AND FIBERS 


It has been well documented that glass rods tend to break in such a 
way that the fracture face comprises three regions known as the mirror, 
the mist, and the hackle zones.!°!! The mirror zone is an optically 
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Fig. 1—A typical glass fiber fracture. 
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smooth surface adjacent to the fracture origin, the hackle zone corre- 
sponds to an area where the fracture has forked and the specimen is 
separated into three or more pieces, and the mist zone is a transition 
region between these two zones. Such behavior is also observed with 
glass fibers. Figure 1 shows the fractured end of a 125-um glass fiber 
which clearly exhibits these three regions. 

It has been experimentally demonstrated" that the distance from 
the origin of fracture to a point on the boundary between the mirror 
and mist zones, r, is given by 


ZVr = K, (1) 


where Z is the local stress at the point in question and XK is a constant 
for a given material. 

A theoretical justification for eq. (1) can be given. Anderson” gives 
the energy balance equation for a crack of length 2c propagating in a 
brittle isotropic material subject to a — stress, Z, as 


2 2 
az (- roe 48 5 kipe Ge + 4c) = 0. (2) 


Here £ is Young’s modulus, p is the density, and y is the surface tension 
of the material. The parameter k is a geometrical factor which depends 
on the shape of the crack. The three terms in eq. (2) represent, respec- 
tively, the released strain energy, the kinetic energy associated with the 
moving crack, and the surface energy of the newly created surfaces. 
As the crack propagates, more and more strain energy is converted 
into kinetic energy until the crack reaches a limiting velocity, é = v,, . 
where v; is roughly 4 the longitudinal sound velocity for the material 
(see, for example, Roterenée 12). At this point the excess energy begins 
to be taken up by the creation of subsurface cracks (the mist zone). 
When the released strain energy is sufficient to create four new sur- 
faces, a hackle zone is created. Thus, at the boundary of the mirror 
and mist zones, 


d a 2 2772 
5,(-7 cali 5 on + 4yc) = 0. (3) 
By differentiating, we find 
Ze = —— oF = a constant ; (4) 


which is of the same form as eq. (1). A similar derivation is given in 
Reference 11. The value of the constant K in eq. (1) is found experi- 
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mentally to have the value 6.1 kg/mm? for soda-lime-silicate glass 
and 7.5 kg/mm for fused silica, in reasonably good agreement with 
the value found" from the evaluation of the constant from eq. (4). 

In order to break an optical fiber in such a way that the mirror zone 
extends across the entire fiber, it is necessary to have the stress at all 
points within the fiber low enough so that ZVr < K. The required 
value of Z at the origin of the fracture depends on the size of the crack 
or flaw from which the fracture originates.” The value of Z cannot be 
allowed to become zero or negative at any point across the fiber, or 
the crack will cease to propagate or propagate in a direction which is 
not perpendicular to the axis of the fiber. Under these conditions, a lip 
is formed on one fiber end. We see, then, that, to make a reliable clean 
mirror zone fracture, the stress distribution across the fiber must be 
suitably adjusted. 


Ill. THE FIBER BREAKING MACHINE 


In the preceding section, we have given the conditions necessary 
to create a mirror zone fracture across an entire fiber end. To deter- 
mine experimentally the range of stress distributions over which clean 
mirror zone fractures can be obtained, an apparatus was constructed 
which could simultaneously bend the fiber and place it under tension. 
In this way, the stress distribution across the fiber can be varied, as 
shown in Fig. 2. For a given average tension (force per unit area), 
T, the stress distribution across the fiber depends on the radius, R, 
of the form over which the fiber is bent. (We assume no shear friction 
between the fiber and the form.) In fact, the stress across the fiber, 
Z(x), is given by 


Za) = T+ 24 = 2), (5) 


where TJ is the average tension on the fiber, H is Young’s modulus, 
and a is the radius of the fiber. 

If R = «, the maximum diameter, dy, of fiber that can be fractured 
with a mirror zone across the entire surface is given by 


Z'Ndu = K, (6) 


where Z’ is the stress necessary to initiate the break. 

In the experiments to be described later using a diamond or carbide 
scorer to initiate the break, we find Z’ } 25 kg/mm?. Thus, for fused 
silica fibers we find dy 100 um and for R = o, when fracturing 
fused silica fibers with diameters 2 100 um, we expect hackle to appear. 
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Fig. 2—(a) A glass fiber bent over a form of radius # and subjected to a tension 7’. 
(b) The stress as a function of position in the fiber for various bending radii 2. 


Our experiments showed this to be the case. If we compute the stress 
from eq. (5), assume T is adjusted so that the stress at x = 0 is Z’, 
and select R (= Ro) sothat Z = Oatx = 2a, we find that the maximum 
value of ZVr occurs on the surface of the fiber at the position where 
r = (V4/5)a, and if we require this product to be <K, we find 
dy(R = Ro) = 3.50 dy (R = ~). Thus, using this technique, fused 
silica fibers of up to ~850 wm in diameter can be fractured with clean, 
mirror zone ends. 

The fibers used for the experiments reported here were multimode 
silica glass fibers with an outer diameter of 125 um and a core diameter 
of 80 um. Ro can be found from eq. (5), letting Z = 0 at x = 125 um, 
assuming the stress necessary to initiate the break, Z’, to be equal to 
the experimentally determined value of 25 kg/mm?, and using the 
values # = 7.2 X 10° kg/mm?; K = 7.5 kg/mm? appropriate for 
silica glasses. We obtain Ry = 3.7 em. 
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Fig. 3—A semi-schematic view of the fiber breaking machine. 


Figure 3 shows a semi-schematic view of the device used to investi- 
gate the fracture properties of optical fibers. The fiber to be broken 
is clamped by clamps No. 1 and No. 2 and slides freely under the 
Teflon*-coated fiber guide. A Teflon-coated form of suitable radius R 
can be raised to cause the fiber to conform to the form by adjustment 
screw A. The tension on the fiber is measured by a tension gauge, which 
measures the mechanical displacement of a stiff steel bar on which 
clamp No. 2 is mounted. The tension can be adjusted with the adjust- 
ment screw B. A scoring blade can be lowered onto the fiber by adjust- 
ment screw C and pulled across the fiber by adjustment screw D. The 
pressure on the scorer blade can be adjusted by changing the weight 
in the counterbalance. 


* Registered trademark of Dupont Co. 
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IV. EXPERIMENTAL RESULTS 


Breaks were made on samples of a low-loss multimode silica glass 
fiber having a core diameter of 80 um and a cladding thickness of 
22 um. A wide range of breaking tensions and fiber-bending radii was 
studied using the fiber-breaking machine described above. We used a 
variety of scoring techniques and attempted breaks in atmospheres 
of various relative humidities. The results can be summarized as 
follows: If the radius of curvature of the form was less than about 
2 cm, a lip would be formed. When fractures were made without using 
a form, i.e., R = © ornegative, a hackle region was produced. ‘‘Good,”’ 
clean fractures were obtained when a 5.7-cm radius of curvature form 
was used. These results are illustrated in Fig. 4. 

Using the 5.7-cm radius of curvature form, clean fractures with no 
visible hackle or lip were always produced using breaking tensions 
in the range of 125 to 175 g and scorer pressures ranging from 1.5 to 
7.0 g. The smallest scores were produced when a sharp diamond 
scorer* was lowered onto the fiber after the tension had been applied. 
We found no effect on the fracture characteristics when the relative 
humidity was varied from 7 to 100 percent, or even when water was 
applied to the point of fracture. In all, a total of 33 fractures were made 
within this range of conditions. /n no case was there any visible evidence 
of any hackle or lip. In the worst case the disturbed region associated 
with the score extended over a distance of ~22 wm. As the cladding 
thickness on this optical fiber was 20 um, this means that in all cases a 
perfect mirror zone fracture occurred over essentially the entire core 
region of the fiber. 

To establish the minimum splice loss in joining such fiber ends, we 
used the setup shown in Fig. 5. The joints were made from ends ob- 
tained from the same fracture, but rotated with respect to the original 
fracture position. In this way, the time between fracture and joining 
was kept at a minimum in order to avoid contamination of the ends. 
Moreover, utmost accuracy was achieved by comparing the losses 
immediately before fracture and immediately after joining. This time 
was typically 10 minutes, while instabilities in the setup caused a 
power drift at the detector of not more than } percent in 30 minutes. 
Joining adjacent ends, of course, eliminated the possibility of diameter 
discrepancies which would be encountered in practical splices. 


* The diamond scorer was supplied by Victory Diamond Tool Co. of East Hanover, 
~d. 
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Fig. 4—Electron microscope photographs of 125-um diameter silicate glass fibers 
broken using various form radii (2): (a) R = 0.75 em; (b) R = © or negative; 
(c) R = 5.7 cm. 


Fibers of the type that were used for the splice loss measurements 
reach a steady-state power distribution after a certain distance inde- 
pendent of the injection conditions. This distribution was measured 
for the fiber in question at the end of a 1.2-km length. The power 
distribution in the splice should preferably be the steady-state dis- 
tribution. Since a sufficient fiber length to achieve such conditions 
was not available for our measurements, we approximated as well as 
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Fig. 5—Schematic of apparatus used to measure laboratory splice loss. 
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possible the steady-state distribution at the input by properly focusing 
the input beam onto the fiber front face. Specifically, we made the 1/e? 
width of the Gaussian field distribution in the input cone equal to the 
1/e? width of the steady-state far-field distribution measured at the 
end of the long sample (0.14 rad half-width). The distance between 
input and splice was 1 m. 

If the splice disturbs the power distribution substantially, a sufficient 
fiber length must be provided after the splice to allow the distribution 
to settle, a process which generally is associated with some excess loss. 
To study the magnitude of this effect, we measured several joints with 
fiber lengths from 1 to 5 m after the Joint. We did not find a consistent 
increase in loss as the length was increased, although 5 m is admittedly 
not a sufficient length to reach the steady state. Further study is 
necessary to estimate the error involved. 

To make a good joint, the ends were aligned using a microscope to 
within a fraction of a degree in angle and within 1 um laterally, but 
kept apart by at least 10 um to avoid damage of the ends by mutual 
abrasion. A drop of glycerin was then added, which was held between 
the ends by surface tension. The refractive index of glycerin is 1.473 
and almost coincides with that of silica glass (n = 1.458). This pro- 
cedure invariably produced a splice with a loss of less than 1 percent 
(typically, 0.5 percent). This result was unaffected by rotating one end 
with respect to the other. Note that no information on the transmitted 
optical signal was required to achieve this optimal alignment. 

To establish the order of magnitude of alignment tolerances per- 
missible in practical splices, we measured the increase in loss as a 
result of longitudinal or lateral misalignments. The fiber ends could 
be parted axially by 100 um (one core diameter) before the losses in- 
creased by 1 percent. Lateral displacements were more critical. The 
loss increased by 1 percent for a 5-um displacement and by 4 percent 
for a 10-um displacement (10 percent of the core diameter). 


V. SUMMARY AND CONCLUSIONS 


We have presented a theory of glass fiber fracture which allows us 
to design a machine for reliably producing clean breaks which leave 
the fiber ends in a suitable condition for splicing. We have built such 
a machine and demonstrated that, with 125-ym silica glass multimode 
optical fibers, such breaks are consistently obtained. Laboratory 
splicing experiments using fibers broken with this machine always 
produced splices with losses of less than 1 percent (0.04 dB). 
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Overload Model of Telephone Network 


Operation 


By R. L. FRANKS and R. W. RISHEL 
(Manuscript received June 5, 1973) 


An analytic model for the steady-state behavior of an overloaded tele- 
phone network is given. The model includes trunk and machine congestion, 
retrials, “‘don’t answer and busy,’’ and some network management controls. 
It is significantly cheaper to use than Monte Carlo simulations for moderate 
size networks. It compares well with Monte Carlo simulation calculations 
of point-to-point completion probabilities and the expected number of 
messages 7n progress. It compares less well for sender attachment delay 
and probability of tume-out calculations in switching machines. 


I. INTRODUCTION TO THE PROBLEM 


The purpose of this paper is to develop an analytic model of a tele- 
phone network which displays the major steady-state behavior of the 
network under overload conditions and which is computationally 
tractable. Besides being used to predict steady-state network opera- 
tion in the presence of overload, such a model should help in the de- 
velopment of insight into network operation. Also, with an analytic 
model available, optimization theory can be brought to bear on various 
problems in network management. 

Analytic network modeling seems to have been aimed at the trunk- 
ing network design problem in the past. The usual approach was to 
assume that switching machines had enough capacity to have no 
effect on the traffic through them. Under these conditions, the stream 
of call attempts on a trunk group may have its distribution changed 
in two ways: the calls have previously been offered to a different trunk 
group and overflowed to the present one, or some of the calls in the 
stream were removed as a result of blocking on a trunk group in series 
with the present one. These cases have been handled by Wilkinson’s 
Equivalent Random Method! and Katz’s Carried Equivalent Method.’ 

When a network is overloaded, the effect of machine congestion is 
not negligible and must be taken into account. Early work on toll 
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machine congestion was done by Helly,? who considered a homogeneous 
group of identical machines connected by infinite trunk groups. His 
approach suggested the way we treat sender holding time in the 
switching machine model. Recently Szybicki‘ gave a model for an 
overloaded local switching machine. 

Monte Carlo call-by-call simulations have been used to study net- 
work behavior. Recent examples at Bell Laboratories are simulations 
by J. A. Kohut’ and J. M. McCormick. These simulations have the 
advantage of great flexibility. They also give transient response as well 
as the steady-state response of the network. Call-by-call simulations 
may require many runs, or long runs, to obtain reliable statistics for 
a process under study. They tend to be more expensive to run than 
analytic models. 


II. INTRODUCTION TO A TELEPHONE NETWORK 


From a traffic point of view, the network consists of end offices, 
switching machines, and trunks. The end offices serve as sources and 
destinations of calls. The trunks are message paths through the net- 
work. The switching machines are nodes in the network at which the 
choice is made of the path to be taken. 

To illustrate the important effects in network operation, let us trace 
the progress of a typical call through the simple network shown in 
Fig. 1. The call enters the network through end office 1. It finds a free 
circuit on the trunk group from 1 to 2, attaches to it, and simul- 
taneously bids for a sender in switching machine 2. After a short wait, it 
is accepted into machine 2. It finds the trunk group from 2 to 4 full, and 
attempts to attach to the trunk group from 2 to 3. There is a free circuit 
on that trunk group, so the call attaches to it and bids for a sender in 
machine 3. This process continues until the call enters end office 5. 
If the destination telephone is not busy, it rings. If it is answered, 
the attempted call is successful and becomes a message. 

This typical call went through three switching machines. The block 





1 
Fig. 1—Network used to show progress of typical call. 
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diagram in Fig. 2 shows the sequence of operations in a Bell System 
No. 4A-ETS switching machine. 

A eall coming into a switching machine enters a queue to wait for a 
vacant sender. When a call gets a sender, its destination information is 
impulsed to the sender, and the sender it had in the previous machine is 
released. 

The call then queues for the remaining common control equipment, 
indicated here as a decoder-marker combination. This decoder-marker 
decides on the machine the call should be routed to next, tests for a 
vacant trunk, and sets up the connection, if possible. If there are no 
vacant trunks to appropriate subsequent machines, a _ no-circuit 
announcement is given. After a no-circult announcement, the trunks 
on all the links over which the call had progressed are released. If the 
call is routed to a subsequent machine, it enters a queue for a sender 
in that machine. 

The sender in the current machine is occupied by the call from the 
time it begins processing the call until the call has transmitted its 
destination information to the sender which processes the call in the 
subsequent machine. If a call waits longer than a fixed time to get 
a sender in the subsequent office, it is timed out. If it is timed out, 
the call is sent back to the marker-decoder which then connects it to a 
no-circult announcement. 

Under normal network operation, very few calls receive a no-circult 
announcement, and fewer still time out while waiting for a sender. By 
far the most important causes of a call failing to become a message 
are for the called telephone to be busy when the call arrives and for 
the called customer to fail to answer the phone when it rings. When 
the network is overloaded, the number of no-circuit announcements 
increases, and time-outs become more frequent. Not only does the 
percentage of failures increase as the network becomes overloaded, but 
the number of successful attempts may actually decrease. 

The factors underlying the decrease in the number of calls carried 
by the telephone network as it becomes highly overloaded were al- 
ready understood in early work, such as Reference 8. As a call is being 
set up, it uses equipment in one switching machine until the next 
switching machine on its route accepts the call and receives the 
destination of the call from the previous machine. If a switching 
machine becomes overloaded, machines adjacent to it will have to 
wait longer to have their calls accepted and the destinations passed on. 
This causes an increase in the service time for putting a call through 
these machines. This in turn may cause the adjacent machines to 
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become congested. The time-out mechanism helps to relieve this con- 
gestive phenomenon. However, even with time-outs, switching machine 
congestion can back up throughout the network. Calls being set up 
occupy trunks on the partial route over which they have progressed. . 
If a large number of calls are attempting on routes which are blocked, 
a portion of the capacity of certain links could be used by these in- 
effective attempts trying to set up. This would use capacity that 
could be utilized by talking calls. These ineffective attempts also use 
switching capacity in machines preceding the blockage. Most blocked 
attempts try again; these retrials increase the congestion. 

The model to be set up will incorporate the features mentioned 
above. Based on those observations, the model must take into account 
both trunking congestion and switching machine congestion. Any 
call which enters the network will take trunk and machine capacity, 
even if it fails. For this reason, its effect on the network depends not 
only on whether a call succeeded or failed, but also on how far it 
progressed and over which particular route. This information must 
also be included in the model. 


Ili, INTRODUCTION TO THE MODEL 


Our view of modeling the network is probabilistic. We assume that 
for each trunk group there is a probability that an attempt on it will 
find a free circuit and for each switching machine there is a probability 
that an attempt on it will be accepted before timing out. We further 
assume that these acceptance probabilities are independent of the past 
history of the attempt. 

The model has two conceptually distinct parts. First, the global 
problem is, given these acceptance probabilities, to find the various 
quantities of interest such as the expected number of messages in 
progress between each source-destination pair, the attempt rate on 
each trunk group and on each switching machine, and the point-to- 
point completion probability. Second, the local problem is, given those 
quantities, to find the acceptance probabilities for each switching 
machine and each trunk group. The local and global problems to- 
gether give a large number of coupled nonlinear equations which de- 
scribe the steady-state behavior of the network. These equations 
form the model. 

For the model to be computationally tractable, the number of equa- 
tions involved must be as small as possible. For this reason we have 
assumed that each stochastic process in the model can be described 
by a single parameter. (For example, the call origination process 
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between a given source-destination pair is assumed to be Poisson.) 
Even with this assumption, the number of equations involved is very 
large for any reasonable size network. 

It must be emphasized that this is a first-generation model for an 
overloaded network. While the model does very well at predicting 
certain statistics, it is relatively poor at predicting others. A model 
using two parameters to describe each stochastic process could be 
more accurate than this one. Within the structure of this model, the 
switching machine treatment could be improved. 

The network model given here is in several ways similar to the 
model used in the optimization problem of Reference 6. 


IV. GLOBAL ASPECTS OF THE MODEL 


This section deals with global, or network, effects caused by local 
phenomena. An example of such a global statistic is the mean number 
of messages in progress between a source-destination pair, which 
depends on various local effects such as the probability a call offered 
to each trunk group will be accepted onto it. 

Before proceeding further, some definitions are required. 

A complete route, R = (a, bi, b2---ba, c), is a list of the switching 
machines through which a call may pass in going from the end office 
connected to a machine a to the end office connected to machine c. 
For example, in Fig. 1 there are two complete routes, (2, 4) and (2, 3, 4), 
from end office 1 to end office 5. 

A partial route, r = (a, bi, be---bm; ¢) of a complete route & de- 
scribes the route occupied by a call in the process of being set up and 
its destination. For example, a call on r = (a, by, be, b3; c) started in 
the end office attached to switching machine a, passed through 
machines a, b;, and be, has entered (or is waiting to enter) machine 


b3, and has as its destination the end office connected to machine c. 
Define 


cr = Expected rate that calls on complete route R = (a, a1- + -az, 6) 
attach to the trunk group from switching machine b to its 
associated end office. 

x, = Expected rate that calls on partial route r = (a, a1: + -az; 0) 
attach to the trunk group from az_1 to au. 

zr = Expected rate that calls on partial route r are connected to 
senders in switching machine ay. 

Expected rate that calls on partial route r time out while 

waiting for senders in switching machine az. 


oe 
be 9 
{| 
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Fig. 3—Relationship of 2,, z;, t;, and br. 


b, = Expected rate that calls on partial route r which attach to 
senders in switching machine a; are blocked because of a 
lack of outgoing circuits. 

SM; = the zth switching machine. 


To make the meaning of these variables more obvious, consider Fig. 3. 
Calls on partial route r = (a, 7, 7; b) attach to trunk group 2j, and 
therefore bid for senders in SM, at a rate x,. Some of them are accepted, 
at arate z,, into SM; and the remaining ones time out at a rate f,. 

Those calls accepted into Si; then attempt to reach end office b by 
attaching to link 7k, at rate x,,, where r1 = (a, 2, j, k; 6). 

If z,, < 2,, some of the calls alternate route over link 7/, with rate 
Lr, Where 72 = (a, 1, j, €; b). If x,, + 2,, < zg, and there is no further 
alternate route available, then some of the calls are blocked with rate 


b, 42, — (2, + 2,,). 


The global aspect of the network relates the z,’s, z,’s, b,’s, and t,’s 
for all partial routes in the network. The following assumptions are 
made: 


(t) The model assumes that every type of call attempting to enter 
switching machine 7 has the same probability, P:, of being 
accepted. Therefore 

ar = P 2, (1) 


for each partial route r entering SM;. 


(7) It further assumes that every type of call attempting to attach 
to trunk group, 27, has the same probability Pa of attachment. 
The resulting equations are given below as (2) for the case 
with no network management control. Appendix A contains 
the equations that result when network management controls 
are included. 
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If a call from a to b in SM; has a preferred route over link 
27 and a second most preferred route over link zk, then 


C. =F pe, 
eae (2) 
Lry = Paes = Tex) 
etc., where 
= (a,t; b) 
>= (a,2,9; b) 
re = (a,t,k; b). 
(17) It is assumed that any call successfully attached to the trunk 
group leading to its destination end office, b, has a probability 


PA, of being answered and therefore becoming a complete 
call. This probability depends on the destination. Therefore, 


LR = PA +P Rep, (3) 


where 


k= (a,2,7,k,b) 
ie (a,2,7,k,b; b). 


(iv) The stream of original calls attempting to go from a to 6 is 
Poisson with mean ).,». Calls that do not complete retry with 
probability PR. The expected long run attempt rate for calls 
from a to b is 


Aa,b = Na,b + PR(Aa,t 7? Cab) 


where Cz.) = a XR, 1s the completion rate for all calls 
R froma tob 


from a to b. That is, 


hab — PR > LR 


Aa, 5 = Ratob (4) 
1— PR 
On the link from end office a to its switching machine, SM,, 
La;b) = PaAa,b- (5) 


Clearly, with assumptions (7)—(iv), all the rates involved can be 
found from the P’s. A method for finding the P’s in terms of the 
x,’s and 2,’s is discussed in the next section. 
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V. LOCAL ASPECTS OF THE MODEL 


The assumptions made in Section IV for the global portion of the 
model place few constraints on the local part, requiring only certain 
acceptance probabilities, P;, to satisfy those assumptions. 


5.1 Toll Machine Model 


The toll machines to be modeled are No. 4A-ETS switching ma- 
chines. The statistics of interest in the model are the acceptance 
probabilities, P, mentioned in the last section, and the expected wait- 
ing time, 7'., for a call to get a sender. For relatively light loads, these 
statistics behave as though the machines selected the next call to be 
served on a first-come, first-served basis.’ When the machines are 
overloaded, they behave as though calls were selected at random for 
service.® 

We calculate the expected waiting time based on the former when 
loads are light and on the latter when they are heavy. The switch 
from the first to the second method is made where the curves of waiting 
time versus load cross. 

In the light-load case the assumptions are 


(2) The stream of calls attempting to enter each switching ma- 
chine is a Poisson stream with mean ),. 
(72) The time a sender is held in SM; by a call is an exponentially 
distributed random variable with mean 1/y;. 
(272) All toll machines have the same time-out interval, 7. 
(zv) The queuing discipline is first-come, first-served. 


The problem of finding the probability, P;, of acceptance of calls 
into SM; under these assumptions has been solved.’ The result is 





I IN 
See fake ce NaN ce a Ss, oe Ss, 
als le | BU Nips) As — Toa | 
Ai = 
+ We ; for \; ~Naz,ui (6) 
P= : T\ , for s=N 
= — | garcw5 + éT| ) or i= LV s;Mi 
where 
N,, = the number of senders in machine 2 
n/n! 
B(n, a) = ual / —, the Erlang B function. 
ir 
j =0 J: 
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Under these assumptions Reference 9 also gives the expected waiting 
time to get a server. The result is 





N , pet Negti-s) _ LN ash: a Nil (Nui — Ax) ] 

if — _— . Ee ee 

eG (Nejme — di)? 
for A; ¥ Nap; (7) 

1. = (1 = P,;)T (1 + SS) ; for Ne = Na;Mi- 


In the heavy load case, assumption (iv) is replaced with 
(iv)’ The queuing discipline is random. 


An asymptotic expression for waiting time under this assumption 1s 


T.,, = T(1 — P./2), hi > Nop. (8) 
Fitting eqs. (7) and (8) together 

The Ae S Noni 
f igi = (9) 

min{ 7%, T,,}, he > Nope. 
To find P; and T.,, all that is needed are T’,, \;, and u;!. T is given and 
i= dD 2, (10) 

rel; 


where 
I; = {r|r = (a,a1---7;b) for some a, 6 and ay::-}. 


To find y;', consider Fig. 4. Calls on partial route r bid for a sender 
and connector in SM, at rate x,. All calls waiting to enter SM; have 
an expected waiting time, 7y,. Calls on partial route r are accepted 
into SM; at rate z,. It takes T'¢ seconds to connect an incoming trunk 
to a sender. 

Once a sender is connected, it requires Tp seconds to pulse the digits 
into that sender. It then waits Twy seconds for a translation device. 
The time to translate the digits, look for an available trunk on an 
acceptable route, and connect to that trunk is taken as a constant, 
Tu seconds. If no circuit is available, a call is attached to a no-circult 
announcement. Otherwise it is attached to a trunk connected to some 
switching machine, say SM;. The call, and the sender in SM,;, wait 
for a sender and connector in SM;. If the call is accepted into SM, it 
takes 7’¢ seconds to connect to the next sender and another 7’p to 
pulse its digits into that sender. If the call hasn’t been accepted by T 
seconds, it times out, and must return to the common control equip- 
ment to be connected to a no-circuit announcement. 


t 
TO OTHER SM’S 


X Z 
r r 
WAIT CONNECT PULSE WAIT Dam WAIT CONNECT PULSE 


TIME—OUT TIME—OUT 


eT | 


SENDER HELD IN SM, 


Fig. 4—Flow diagram for expected sender holding time. 
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The expected time a sender is held by any call accepted into SM; is 


we = (Te + Twu + Tm) 
+ (Expected waiting time for calls leaving S//;) 
+ (Twa + Tx) (Probability a call leaving SM; times out) 
+ (Tp + Te) (Probability a call leaving SM; is accepted 


into the next machine). 
More explicitly, 


> Tw,0r 
wt = (Te+Twu + Tu|t+ | 
reli 
1 =. 2d, @r 7 Pes, oF 
+(Twau + Tu] een a a 
rels 
eS er Si dR 
+[(Tpe4+ Te] ar aaa , (11) 
reli 
where 
ae —= OY 
when 


r = (d,a1---ag; b) 


and 0; and 0; are sets of partial and complete routes, respectively, 
defined by 


0; = {r|r = (a,a1:--t,a¢; b) for some a, b, a1---} 
0; = {R|R = (a,a1---i,b) for some a, b, a1: ++}. 


The only symbol in (11) remaining to be explained is Twa, the 
expected time spent waiting for a decoder-marker. The decoder- 
markers are modeled as a finite source queue. The assumptions are 


(z) There are N., exponential servers. 
(22) The queuing discipline is first-come, first-served. 
(iit) Each N, sender in the switching machine either is waiting for 
or receiving marker service or is generating its next marker 
bid with an exponential interarrival time of mean 1/y. 


This finite source queuing model has been analyzed in Reference 10. 
The result, in a convenient computational form thanks to D. Jagerman, 

1s 
T _ Tm N,—M, — ALi — BIN. — Nu — 1, A) ] 
WM —~— yr 


Nom Nd) = (—t)? 
Nm 44 B(N, —Nm—1,A) Nie wl 
i=1 (yT m1) 


(12) 
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where the subscripts corresponding to the switching machine have 
been suppressed and 


Nin 
ae 
NOZENN span =e) 


1 
2S is ee 
NUN =P 1) see = 1) 
To compute T ww, it is necessary to know y, which depends on the 
mean rate, m, at which calls arrive at the marker-decoder queue 


m= >Y24+ > t- (13) 


reli REO; 
Making use of Little’s Theorem" and the definitions of m and y, 


NO 


a er es 
7 Nees m(Tu + Twa) 


Equations (13) and (14) have a unique solution for all rates, m, 
which can be handled by the machines. 


(14) 


5.2 Comments on the Switching Machine Model 


The interaction of switching machines in the real network is known 
to be an important cause of congestion. That interaction is included 
in this model by the waiting time of senders in one machine affecting 
the holding time of senders in adjacent machines. 

This model has some features that appear to be ad hoc. The assump- 
tions, however, are computationally convenient and give results 
similar to gross machine behavior. The network model has been 
structured to accept expanded machine models if they are required. 

Section VII, on validation, includes a discussion of the accuracy of 
this switching machine model. 


5.38 Trunk Group Model 


The probability that an attempt will be accepted on a trunk group 
is found by assuming that the arrival process is Poisson. The required 
result 1s the Erlang B function, which depends only on the mean 
number of calls which would be on the trunk group if it were infinite 
and the actual number of trunks. 

To find P73 on the trunk group between 7 and j, we need the following 
definitions : 


Nz = number of trunks between SM; and SM;,. 
Hy = expected number of calls on the trunk group between 7 and 7. 
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Fy = expected number of calls that would be on the trunk group 

if Nz were infinite. 

nr = expected number of messages on complete route R. 

S, = expected number of calls on partial route 7 being processed 
in Si; where r = (a---é; b). 

W, = expected number of calls on partial route r waiting to enter 
SM; where r = (a---4; b). 


I 


| 


= mean holding time for a message. 


Ly = > _met+ > _S,+ ~ _W, 


R over tj r over 1j r over 7j 


1 
V R over ij 


+- y, {T we > Lr + (Tc + T p + Twa SS T mu) » Zr} 
all TC, rely rel ¢_ 
r over ij r over 17 


+ (Twa + Tm) dD _#, (15) 


els IE 
To find Fy, simply notice that P7 is a linear factor of every 2,, Z,, 
and t, in eq. (15). Given all the P’s except Pi, choose any positive 
value for Pz, say Pa, use eqs. (1) through (15) to find £7, the value 
of Hz corresponding to P35. Then 





By 
Fg = ee (16) 
Finally 
PZ = 1 — BING, Fa), (17) 


where B is the Erlang B function. 

It is unlikely that the arrival process at each trunk group in the 
network is Poisson. In fact, much of the recent trunking analysis has 
been directed toward non-Poisson processes. However, the Poisson 
assumption is a reasonable one to make in a mean value model. The 
accuracy of the overall model will be discussed in Section VII. 


VI. SOLVING THE EQUATIONS 


In the last two sections the model was given as a set of equations, 
(1) through (17). Most reasonable uses of the model require simul- 
taneous solution of the equations and then computation of quantities 
of interest from this solution. 
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Fig. 5—Network used in point-to-point comparison run. 


Solving these equations, at reasonable cost, is essential to the use- 
fulness of the model. As given in Sections IV and V, there are a very 
large number of equations, both nonlinear and coupled. In the network 
shown in Fig. 5 are about 50,000 equations and variables. Fortunately, 
the P’s and T',’s calculated in Section V form a fundamental set of 
variables from which everything else can be calculated. There are 91 
of these variables for the network in Fig. 5. In another network of 
interest there are 240 of these variables. 

The approach taken to solving the equations is iterative. Given a set 
of P’s and T,,’s, all other variables of Section IV are calculated. Then 
a new set of P’s and 7'...’s are calculated. If the new set and the old 
set are the same, a solution has been found. More precisely, let y be a 
vector whose components are the P’s and 7',’s, then the equations of 
the model specify a function, F(y), which gives the new value of P’s 
and 7',’s. In this framework solving the model equation is equivalent 
to solving 

F(y) = y. (18) 

To make the solution of (18) easier, the components were normalized 
to the interval [0, 1]. The components corresponding to P’s are 
necessarily in this interval. The components corresponding to T'~’s 
were forced to be in the interval by replacing Ty, by T'y,/T. The re- 
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sulting function F maps [0, 1 ]” into itself, where n is the number of 
trunk groups plus twice the number of toll machines. The function is 
continuous so (18) has a solution by Brouwer’s Fixed Point Theorem.” 
The question of uniqueness of the solution will be discussed later. 

For this model to be a useful tool for network analysis, it 1s necessary 
to solve (18) inexpensively. There are two basic problems to be over- 
come. First, F is so complicated that, for reasonable size networks, its 
derivative is unavailable. This means that any method which requires 
F’ cannot be used. Second, in these same networks a single evaluation 
of F costs on the order of $0.50. The cost of estimating F’ by n evalua- 
tions of F depends on n, the dimension of y. For the network in Fig. 
5 it would cost about $45 for a single estimate of F’. 

In order to solve (18) economically, an algorithm which doesn’t 
require F’ or estimates of it was devised. The algorithm adapts the 
step size on the basis of the last ten evaluations of F. 

The algorithm is as follows: 


(z) Initialize y’«[0, 1 |]* and set 2 = 0. 
(i) Lf ||Fy* — y'l] < e, stop. 
(222) Otherwise, 
OS jen 1 + 8; 
y'+a(Fry’ — y’), 


yor 


where 
I; = {j|j an integer = 0,7 => 7 = 7 — 10} 
do — 1 
_ Wy? — y4)— (Fy* — y* 
3 
lly? — y*"|| 
(2v) Repeat (72) with: =71+ 1. 


7> 0. 


A computer program was written to evaluate F'(y), ie., eqs. (1) 
through (17), and to implement the algorithm for solving (18). Our 
experience with the program has been that the algorithm usually 
reaches a satisfactory solution in less than n/2 steps. The cost of the 
program depends on the network size and on the value of various calling 
parameters. However, the cost for the network shown in Fig. 5 was 
usually around $10, with some runs as high as $40. For a larger net- 
work with 240 variables, the cost was usually around $20. 

An important point to mention is that F is not a contraction map- 
ping. This means that the existence of a unique solution cannot be 
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Fig. 6—Example network which has two quasi-stationary solutions. 


guaranteed. In fact, in one example, two solutions to (18) were found. 
The question immediately arises, ‘‘Are both of these solutions physi- 
cally meaningful?” The answer is ‘‘Apparently, yes.’’ The existence 
of two stable operating regimes for the toll network has been suggested 
before.’ The argument for their existence is as follows: If large queues 
form before switching machines, other machines will have their hold- 
ing times greatly increased and will be able to switch only a fraction 
of their usual capacity. All other calls will time out. That means, if 
somehow the queues become large, they might stay large and the net- 
work completion rate would be very low, while under the same system 
parameters if the queues ever became small, they would stay small 
and the network completion rate would be much larger. 

To test this argument, J. A. Kohut’s Monte Carlo Simulation® of 
the network shown in Fig. 6 was run. A brief description of this simu- 
lation is given in Appendix B. The system started empty and was run 
for one hour of simulated time, reaching a quasi-stationary condition 
within the first 10 minutes. Then the offered load was doubled for 10 
minutes, causing large queues to form. After that, the simulation was 
run for one more hour at the original traffic level. Again it reached a 
quasi-stationary state within 10 minutes. The results are given in 
Table I. The results show conclusively, for this simulated network, 
that two quasi-stationary operating regimes exist. For comparison, 
the results from the analytic model are also included in Table I. 

It is possible for this network to be operating in the uncongested 
regime, receive an unusually large number of calls during some short 
time interval, and go into the congested regime. In the congested 
regime, if an unusually small number of calls arrived for a period of 
time, the system could go into the uncongested regime. It seems 
reasonable, and the simulation run helps confirm it, that the mean 
time before spontaneously leaving one of the regimes is quite long. 
This is the reason for using the term quasi-stationary. 
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TABLE J—HXAMPLE SHOWING Two 
QUASI-STATIONARY SOLUTIONS 


Model Mean Attempts | Mean Time-outs Mean Waiting 
ode (per 5 min. (per 5 min.) Time (in sec.) 
Simulation 
10 to 60 min. 602 + 9 0 0.17 + 0.02 
80 to 1380 min. 605 + 8 454+ 11 21.5 +0.2 
Analytic 
Solution 1 600 10 0.22 
Solution 2 600 459 22.7 


The existence of two quasi-stationary operating regimes apparently 
has implications for network management. If the network is congested, 
it may not be due to high calling rates but only due to high sender 
queues. A control which clears out these queues may be enough to 
decongest the network. Short sender timing which is currently used 
in the network is such a control. 


VII. MODEL VALIDATION 


The model discussed in previous sections contains many important 
network features. The machine model includes the stochastic arrival 
of attempts, office work times depending on waiting for adjacent 
offices, and time-outs. The trunk model includes stochastic arrivals 
and holding times, with the mean holding times dependent on how 
far the calls progressed toward becoming messages. 

In the final analysis, the model stands or falls by how well it pre- 
dicts the operation of a real network under overload. While from a 
validation viewpoint this could best be done by comparing the model 
with a real situation, there are two good reasons not to do so. First, 
the data collection problem would be extremely difficult and pro- 
hibitively expensive. Second, getting meaningful comparisons would 
require allowing the network to operate in an unacceptable mode. In 
addition, any real network would include things not modeled here, 
such as other types of switching machines and additional network 
management controls. 

An alternative is to compare the model with a Monte Carlo simu- 
lation which is currently being used to evaluate network controls. 
While this comparison cannot evaluate the modeling of effects treated 
similarly in both models, it does help evaluate the modeling of effects 
treated differently. We compare our model with Kohut’s simulation. 


OVERLOAD MODEL 1607 


It also contains a simplified machine model, but does not include our 
simplifying assumptions on how senders wait for senders, that all 
arrival processes are Poisson, or that machine holding times are ex- 
ponentially distributed. These assumptions are perhaps the most 
suspect in our model. 

Two kinds of comparison between the two models were carried out. 
The first compares gross behavior over a very large range of offered 
loads. The second looks at more detailed statistics under a reasonable 
overload. In both cases, the results are similar. 

The first type of comparison was carried out on the network in Fig. 
7. The two models were given exactly the same data on machine sizes, 
pulsing times, trunk group sizes, etc. A series of runs was made with 
the only change between runs being the calling rates. The first run 
used a nominal set of calling rates, the second used twice the nominal 
calling rates, the third used three times the nominal calling rates, etc. 
For each run, the Monte Carlo simulation was run until, on the basis 
of the retrial rates, it appeared to be in steady state. It was then run 
for another one or two simulated hours to estimate the expected num- 
ber of messages in progress in the network in steady state. The sample 
variance was used to estimate the 68 percent confidence interval for 
the mean. The analytical model was then used to find the expected 
number of messages in progress for each offered load. Figure 8 shows 
the curve generated by the analytic model as well as the Monte Carlo 
simulation’s estimates of the corresponding means and confidence 
intervals. 

In Fig. 9, exactly the same runs were made as in Fig. 8, except that 
switching machines timed out after 5 seconds in Fig. 8 and after 30 
seconds in Fig. 9. Monte Carlo runs for more than double the nominal 
calling rates were not made in the 30-second case, since the simulation 
was not intended to handle the very large queues that would develop. 

From Figs. 8 and 9, it appears that the models predict the same 
behavior for the expected number of messages in progress over a very 
large range of calling rates. The numerical values given by the two 





Fig. 7—Network used in massive overload comparison runs. 
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Fig. 8—Carried vs offered load, time-out = 5 seconds. 


models also seem to agree well. At three times the nominal calling rate, 
the two means differ by less than 4 percent in Fig. 8. 

The previous examples were generated for the network in Fig. 7. 
The second type of comparison was made between the Monte Carlo 
simulation and the analytic model on the network in Fig. 5. This 
network configuration was used in early network management simu- 
lation studies. While it is similar in structure to the toll network, it 
has one less level of hierarchy. 

In order to get reliable statistics, the Monte Carlo simulation was 
run for three simulated hours. The network appeared to have reached 
equilibrium by the end of the first hour. Statistics were printed out at 
10-minute intervals for the next two hours, and these were used to 
estimate completion probabilities, expected sender attachment delay, 
and the probability a call would time out in each switching machine. 

Table II shows the comparison of the expected sender attachment 
delay and probability of time-out given by each model for each of the 
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Fig. 9—Carried vs offered load, time-out = 30 seconds. 


17 switching machines. For the Monte Carlo run, estimates of the 
standard deviation in the estimates are also given. It can be seen that 
the analytic model tends to give larger numbers for both quantities 
than does the Monte Carlo simulation. 

To estimate the completion ratio for each point-to-point pair, the 
total number of attempts and completions were recorded for each pair 
for the last two simulated hours of the Monte Carlo run. The estimate 
of the completion ratio from source 7 to destination 7 is 


C';; = number of completions from 2 to 7 
A;; = number of attempts from 2 to 7. 


This estimate of the completion ratio was used since it has a smaller 
variance than would result if the completion ratio was calculated for 
each 10-minute interval and then averaged. The corresponding com- 
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TABLE LI—CoMPARISON OF ANALYTIC AND MonrTe CARLO 
MopEL RESULTS FOR SWITCHING MACHINES 


er Attachment Delay (in sec.) Probability of Time-out 
Switching 
Machine 

Analytic Monte Carlo Analytic Monte Carlo 
1 0.33 0.19 + 0.03 0.008 0.008 + 0.002 
2 0.12 0.08 + 0.01 0.002 0.002 + 0.001 
3 0.30 0.20 + 0.03 0.016 0.012 + 0.002 
4 0.25 0.16 + 0.03 0.013 0.007 + 0.002 
5 0.29 0.21 + 0.03 0.015 0.012 + 0.002 
6 0.18 0.12 + 0.02 0.008 0.007 + 0.002 
7 0.10 0.04 + 0.02 0.004 0.001 + 0.001 
8 0.79 0.31 + 0.03 0.011 0.009 + 0.002 
9 0.49 0.19 + 0.03 0.016 0.007 + 0.002 
10 0.15 0.07 + 0.01 0.005 0.003 + 0.001 
11 3.04 2.03 + 0.07 0.219 0.163 + 0.009 
12 1.83 0.90 + 0.06 0.085 0.054 + 0.006 
13 0.46 0.32 + 0.04 0.027 0.020 + 0.003 
14 0.17 0.07 + 0.02 0.008 0.002 + 0.001 
15 0.18 0.12 + 0.02 0.008 0.007 + 0.002 
16 0.19 0.08 + 0.01 0.009 0.003 + 0.001 
17 0.12 0.04 + 0.01 0.005 0.002 + 0.001 


pletion ratio calculated by the analytic model will be denoted CR;;. 
To compare CR,; with CR;;, it is necessary to have an estimate of the 
standard deviation of CR;;. This estimate was made as follows: If the 
actual completion ratio really is CR;; and if the probabilities of com- 
pletion are independent for successive 17 attempts, then given the 
number of 77 attempts, the number of 27 completions is a binomial 
random variable. Therefore, CR;; has mean and standard deviation 


E(CR,;] = CRi; 

CR,;;(1 — CR;;) 
Aij , 

respectively. The assumption that successive completion probabilities 

are independent is not unreasonable, since in this network the trunk 

groups between a typical 27 pair will have an average of 10 to 50 

message completions between successive 77 attempts. 


To conveniently compare CR;; and CR,;, consider the standardized 
random variable 


| 





Cij = 


Es A Chis — CR; (19) 


Otis 


Figure 10 has a histogram of the £;,;’s for 72 arbitrarily chosen 77 pairs. 
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In the pairs plotted, 68 percent of the £;,’s were in [ —1, 1 ], 92 percent 
were in [ —2, 2], and 100 percent were in [ —3, 3 ]. This is consistent 
with the above assumptions. If they were correct, the expected percent- 
ages would be approximately 68, 95, and 99.7 percent, respectively. 

To get a quantitative estimate of the difference in the completion 
ratios given by the two models, we require an estimate of E[é;;]. To 
get such an estimate, treat the £;,;’s as independent, identically dis- 
tributed random variables. Then, using the 72-pair sample to estimate 
E{&:; | and the standard deviation of that estimate gives 


E[E:;] = 0.026 - 0.143. (20) 


This is consistent with E[é:;] = 0. However, using the estimated 
mean allows us to estimate the relative error, ¢:;, between the two 
models. 


€ij = B| CR; (21) 
From (19), 
os Hei), 


Using Eé;; ] = 0.026, ¢:; was calculated for all 72 point-to-point pairs. 
All the calculated values were in (0.0017, 0.0051) and the average was 
0.0033. This gives the estimated relative difference in the completion 
ratios calculated from the two models as 0.33 percent. This error is 
negligible for practical purposes. 


100% (99.7%) . 


92% (95%) 


68% (68%) 


—3 —2 —1 0 1 2 3 


Fig. 10—Relative frequency of é;;’s for 72 point-to-point pairs. 
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From the examples given in this section, it seems reasonable to 
conclude that 


(t) The two models agree well when computing network quantities 
such as point-to-point completion ratios and the expected 
number of messages in progress. 

(it) They show somewhat less agreement in local phenomena such 
as the sender attachment delay and probability of time-out in 
individual machines. 


VIII. CONCLUDING REMARKS 


This paper presented an analytic model of the response of a tele- 
phone network to overloads. The model agrees well with a currently 
used simulation for network quantities such as point-to-point com- 
pletion ratios and the expected number of calls in progress in the 
network. It is much cheaper to use, with typical costs being $20 
versus $200. The model only gives quasi-stationary results. To get 
the transient response, a Monte Carlo simulation must be used. 

The overall model structure permits changes in the model of in- 
dividual components. Expanding the No. 4A switching machine 
model, including other types of switching machines, and including 
additional network management controls might be useful. 


IX. ACKNOWLEDGMENTS 


Many people have contributed ideas to this work. We wish partic- 
ularly to acknowlecge contributions by Jack Holtzman, Sheldon 
Horing, Edwin Messerli, Irvin Yavelberg, Pat Spagon, Dave Jagerman, 
and George Hallas. We also take this opportunity to thank Elizabeth 
Murphy for early programming assistance, and Ellen Hill for produc- 
ing the final program. Finally, we wish to thank John Kohut for help 
in connection with his Monte Carlo simulation model of the network. 


APPENDIX A 
Introducing Some Network Management Controls into the Model 


For ease of exposition, eqs. (2) omitted network management 
controls. These equations must be modified to include network 
management controls corresponding to switching machine code 
blocking on the basis of destination, skip routing, cancellation of 
alternate routing from a trunk group, and cancellation of alternate 
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routing to a trunk group. To see how these controls are included, let 


r = (a,t; da) 
a= (2,2,9;3 d) 
2 = (a,i,k; d). 


Partial route r enters switching machine 7. Partial route r; is the pre- 
ferred route out of machine 7 toward the destination d. Partial route 
ro is the alternative to route 7;. Also let 


NCB,a = one minus the fraction of calls code blocked in switching 
machine 7 because they have destination d 

NCF;; = one minus the fraction of calls cancelled because they 
attempt to alternate route from trunk group 27 


NCT;, = one minus the fraction of calls cancelled because they 
attempt to alternate route to trunk group 7k 

NSK jx = one minus the fraction of calls which skip over trunk 
group 7k when alternate routing. 


In terms of these symbols, the replacements for eqs. (2) are 


tr, = P332,NCB ia (2') 
Lire = PiuNSKi{NCT iN CFi;(2-NCBia = De) ts 


The interpretation of the equations is as follows: The calls entering 
SM, on route r which are not code blocked are offered to trunk group 
27. They are accepted with probability P;;. The quantity in parentheses 
corresponds to calls which are neither code blocked nor accepted into 
route 71. The quantity in brackets corresponds to calls which are not 
cancelled because of alternate routing controls. Of those calls, the 
ones which do not skip trunk group 7k and do find a free trunk enter 
route 72. Those calls which do skip trunk group 7k or find it full will be 
offered to the next alternate route, if one exists. 


APPENDIX B 
A Brief Description of the Monte Carlo Simulation in Reference 5 


The Monte Carlo simulation in Reference 5 is a call-by-call simulation 
in the sense that it generates calls individually and processes them 
through the simulated network as individual entities. That 1s, each 
run of the simulation of Reference 5 produces a realization of the under- 
lying stochastic process as opposed to the model presented here which 
analytically arrives at statistics for that process. The remainder of this 
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appendix describes the assumptions and treatments used in the 
simulation. 

The underlying traffic between each source-destination pair is a 
Poisson stream. Any attempt which reaches its destination end office 
has a fixed probability of failing because of a “‘don’t answer’”’ or ‘‘busy”’ 
condition. Any attempt which fails to become a message, for any 
reason, will retry with a fixed probability. If a failed attempt will 
retry, the time until retrial is chosen from an exponential distribution. 
The conversation length for each successful attempt is also chosen 
from an exponential distribution. 

An attempt which arrives at a trunk group can seize a trunk only 
if one is free at that time. Once a trunk is seized, it is held while the 
attempt progresses through the network. If the attempt fails, the 
trunk is released at the time of failure. If the attempt becomes a 
message, the trunk is also held for the duration of the conversation. 

The simulation contains several switching machine models, only 
one of which was used in the comparisons in this paper. It consists of 
two groups of parallel servers: the first models the senders, while the 
second models the common control responsible for translation, trunk 
testing, and switching. We will refer to the first group as the senders 
and the second as the markers. | 

An attempt bids for a sender, if one is seized, then a constant delay 
is introduced to represent receiving digits. If a sender is not seized 
within a specified time, the attempt abandons the queue. After a 
sender has received the digits, a bid is made for a marker. If one is 
available, it 1s seized and held for a constant holding time. If one is 
not available, the sender will wait for a marker. During the marker 
operation, a test 1s made for a free trunk. If no trunk is available, the 
call is immediately blocked and the sender and all prior seized trunks 
are released. In the simulation, it is assumed that announcements and 
reorder tone do not extend the holding time of blocked attempts. 

After the marker holding time, the sender bids for an attachment 
to a sender at a distant machine. This bid will result in either an 
attachment of a sender or an intersender time-out. In the former case, 
the sender is held for an additional constant length of time which 
simulates outpulsing the digits. In the latter case, no out-pulsing 
occurs, but an additional marker usage is required to route the attempt 
to an announcement. 

The queuing discipline for senders and markers is random. When a 
piece of common control becomes free, a bid is selected at random 
from the bids waiting. The simulated switch of an attempt through a 
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switching machine may encounter delay in four different ways. An 
attempt will be delayed during the sender and marker service times 
and may be delayed by waiting for these pieces of equipment if they 
are not available at the time of the bid. The service time delays are 
fixed, so these delays are equal for all attempts. However, the delays 
caused by queuing are random and are dependent upon how long an 
attempt must wait for equipment to become free. 
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Peakedness of Traftic Carried by a Finite 
Trunk Group With Renewal Input 


By H. HEFFES and J. M. HOLTZMAN 
(Manuscript received May 17, 1973) 


In trunking theory, peakedness 1s defined conventionally as the variance- 
to-mean ratio of a traffic load when carried on an infinite trunk group. 
For analysis of switching machine delays, at has proven useful to define 
a peakedness measure associated with the Carried Arrival Process (CAP), 
the stream of call arrivals carried on an incoming trunk group. The peaked- 
ness of the CAP is defined to be the conventional peakedness of a fictitious 
traffic-load process generated by associating with each carried arrival an 
endependent exponentially distributed holding time with mean equal to 
the mean of calls actually carried on the trunk group. 

The problem considered ts the effect of trunk group congestion on the 
peakedness of the CAP for traffic consisting of renewal inputs offered on 
a blocked-calls-cleared basis to a finite trunk group with exponential hold- 
ing tames. The CAP 1s characterized as a semi-Markov process. This 
model leads to the determination of the peakedness of the CAP. Numerical 
results illustrate the reduction of peakedness, or smoothing, introduced 
by the congestion. 


I. INTRODUCTION 


This paper is concerned with characterizing the traffic offered to a 
switching machine, taking into account both the alternate routing 
that the traffic may have undergone and the smoothing of the traffic 
resulting from congestion on the trunk group incoming to the machine. 
In trunking theory, peakedness is defined conventionally as the 
variance-to-mean ratio of a traffic load carried on an infinite trunk 
group. It is well known that trunk group blocking of peaked traffic, 
such as overflow traffic, can be substantially larger than the blocking 
seen by Poisson traffic with the same intensity. Similarly, switching 
machine* delay and capacity can be quite sensitive to the peakedness 


* Throughout this paper, when we refer to a switching machine we mean the com- 
mon control devices in a switching machine. 
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of the incoming traffic.1 To determine the peakedness of the traffic 
offered to a switching machine, we must take into account the smooth- 
ing effect of the incoming trunk groups. To this end, we consider the 
process of arrivals offered to a trunk group which are carried by that 
trunk group. We call this process the Carried Arrival Process, or CAP. 

To illustrate the CAP, consider the alternate routing network 
shown in Fig. 1. Here traffic overflowing trunk group AB is then 
offered to trunk group AC [Fig. 1(c) ]. Those calls finding free circuits 
on AC then appear at node C as requests for service. The CAP is 
illustrated in Fig. 1(d). 

The basic model used in the analysis is shown in Fig. 2 where a re- 
newal process is offered to a group of N trunks on a blocked-calls- 
cleared (BCC) basis. The renewal input allows us to consider overflow 
traffic offered to an incoming trunk group. The holding times on the 
trunks are assumed to be independent, identically distributed, ex- 
ponential random variables with service rate u. 

For analysis of machine performance it has proven useful to define 
a measure of peakedness for the CAP, z,, equal to the variance-to- 
mean ratio of the traffic load carried (number of trunks occupied) on 
an infinite trunk group to which the Carried Arrival Process has been 
offered. By definition, the holding times on the infinite trunk group 


(a) 


OFFERED 
TO A-B 


(b) 


OVERFLOW 
TO A-C 





(c) 


CARRIED 
(d) | | | | | | | | ON A-C 
(OFFERED 
TO NODE C) 
Fig. 1—Carried Arrival Process. 
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Fig. 2—Carried arrival model. 


are independent, identically distributed, exponential random variables 
(with service rate u) which are independent of the holding times on the 
incoming trunks. * 

The peakedness of the CAP should be distinguished from the 
varlance-to-mean ratio of busy trunks on the incoming group, a 
quantity which is discussed, for example, in Section 8.4 of Reference 2 
for the case of Poisson input. This distinction can be made clear by 
considering an example of Poisson traffic of intensity \(calls/second) 
offered to NV trunks. As \ approaches zero, the two measures approach 
one. Clearly, as \ gets large, the variance of busy circuits in the trunk 
group goes to zero and the mean goes to N, giving a variance-to-mean 
ratio of zero. On the other hand, as d gets large, the time differences 
between successive carried calls approach independent, exponential 
random variables with rate Nu (i.e., a Poisson stream) and the peaked- 
ness 2. approaches unity. This is illustrated graphically in Fig. 3 
which plots z. and (v/m)x.s. (variance-to-mean ratio of busy servers 
on the N trunks) as a function of offered load for N = 10. The example 
is a special case of the general results derived in this paper for arbitrary 
renewal input to the trunk group. 

By modeling the CAP as a semi-Markov process (SMP), it becomes 
possible to calculate peakedness z, as a function of the peakedness of 
the traffic offered to the incoming trunk group and of the congestion 
encountered on the group. The resulting z. may then be used in the 
determination of machine performance.! Numerical results illustrate 
the reduction in peakedness, or smoothing, introduced by the trunk 
group congestion. In the course of determining the peakedness, the 
transform of the distribution of the time between carried calls is 
derived. 


_ “That is, although carried arrivals are accepted simultaneously on the finite and 
infinite trunk groups, the departure times are different. 
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Fig. 3—Distinction between peakedness and variance-to-mean ratio of busy 
Servers. 


II, OUTLINE OF RESULTS 


In this section, we give an informal overview of the results. In 
particular, we give the key equations that a user can employ to deter- 
mine the peakedness, z., of a Carried Arrival Process. The equation 
numbers are the same as will be used in the derivation. Stationarity 
is assumed throughout. In all that follows, we assume unity holding 
time (or the time unit is the mean holding time). 

First of all, 
= M+ -5, 
where M* is the mean number of calls up on the infinite trunk just 
after the time that a call is accepted onto the finite (and infinite) 
trunk group. (1/8) is the mean number of carried calls (on both the 
finite and infinite trunk group).* 


M* is determined from the following equation: 


1 o(N + 1) 
Mt = oy - MaPE| SS | (32) 


where ¢(s) is the Laplace-Stieltjes transform of the interarrival time 


(11) 


‘ Note that, since we are assuming unity holding time, M+ and (1/8) are in 
erlangs. 
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distribution. Mj is the mean number of calls up on the infinite trunk 
group immediately after arrival of a carried call, given 7 calls in progress 
on the finite trunk group immediately following the arrival of the 
earried call. P; (7 = 0, 1, ---, N) is the probability that an arrival 
finds 7 trunks busy on the finite access group, and 


P31 


i eens oe 


Pe (24)* 
is the probability of j calls up on the finite trunk group immediately 
following a carried call arrival. 

The only quantity left to be determined in (82) is M} which is 
calculated by solving the linear equations 


[Mi 2 1 |P;F = > CimnM;, m= 1, ze ve a N, (33) 
1 =max(1,m—1) 
where 
l L—-m+1 = 
Cm= PH 1) & mE) (160 +m 
m— 1 7 =0 n 
m—-1S/1SN-—1, 
l<smSQN, (34)? 
N No=mtl (N — 1 
PiK(, 1) 2 (Oo  ) ieGtm 
= rs ‘ t 


A simple method of solving (338) is discussed at the end of Section VI. 
In the course of deriving the expressions which ultimately determine 
the peakedness of the CAP, the Laplace-Stieltjes transform of the 
distribution of time between carried call arrivals is obtained. This is 
given by 
= [1 — ¢(s) ]o(s + N) a 
OS) = Oe) ay 6G 4N) Px (23) 


Note that the CAP is not completely characterized by (23), since it is 
not generally a renewal process. 
Examples are given in Section VII. 


* One method of computing P; is via the equations given on p. 179 of Reference 3. 
Alternate methods which may avoid some of the numerical difficulties inherent in 
this approach will be discussed in Appendix C. 

T Alternate expressions for special cases, more suitable for computation, are given 
in Appendix C. 
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Fig. 4—Peakedness of the Carried Arrival Process. 


III. CHARACTERIZATION OF THE CARRIED ARRIVAL PROCESS AS A SEMI- 
MARKOV PROCESS 


Consider a renewal process, with a nonlattice interarrival time dis- 
tribution F(t), as an input to N trunks with mutually independent 
exponential holding times each with mean unity (or the time unit is 
the mean holding time). F'(0*) is assumed to be zero. Blocked calls are 
cleared. Such a system is analyzed in Chapter 4 of Reference 38. 

Let {r:,7 = 1, 2, ---} denote the sequence of times at which calls 
are accepted by the N servers (this is, of course, a subset of the times 
at which calls are offered). Let 7(¢) be the number of servers busy at 
time t. Then j(7) is the number of servers busy just after the nth carry 
(nth carried call). Note that P{j(7*) = 0} = 0. It is clear that the 
j(t) process held fixed at j(rt) for rn <t < ra41 is an SMP.* The 
transition probabilities for the embedded Markov chain are derived 
as follows. 

Since there is a death process on the finite trunk group between 
carried calls, we have for m = 1, 2, ---, N and for! =m-—1, :--, 
N — 1 (in which case, the next arrival is the next carry) 


Pij(rr) = m|j(ti1) = UY 


7 I ( | Jomo — et jerk (t) 


anus l—-m+1 | 
= (m1) zr (CO ™**) (p+ m— 1), W) 


where ¢(s) is the Laplace-Stieltjes transform of F(¢), 
6() = fo evar(o: (2) 
0 


*For an introduction to semi-Markov processes, see Reference 4, Chapter 5. 
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When j(r7_,) = N, note that the next arrival need not be the next 
carry. With s.c. denoting service completion and F;(é¢) the zth-fold 
convolution of F'(¢), we have 


P{i(rt) = m| j(ri-1) = N} 


- no s.c. before the first (¢ — 1) 
dD P sarrivals after rr1, (VN — m + 1) s.c.’s 
us before zth arrival after rp-1 


> ce e-Ns ( N Q gear | ae e— (ts) |N-mtl 
t=1 J0 0 tt = 
XdF-1(s)dF (t — s) 


7 =0 
1-3) (8) 


| 


j(r#_1) =N 





Letting Fin(t) be the conditional probability that a transition will 
take place within a time ?¢, given that the process has just entered 1 
and will next enter m, we have 


l 
~(m—-1)t 1 — pot ]l—mt1 
ay [1 — e*] dF (t) , 


P{j(rt) = m| j(7i) = 
bed en, 


re t 
> i e-Ns ( N ) e7 (m—1) (t—8) 
jul J0 m— 1 


X[L1—e (9) X-Fi (s)dF (t—s) 
Pig(ry) = m| iri) = N} 


Although the SMP characterization of the CAP is of general in- 
terest, it is particularly useful in determining the peakedness of the 
CAP as we shall see in Section IV. 


= 


IV. PEAKEDNESS OF THE CARRIED ARRIVAL PROCESS 


Recall that the peakedness of a process is the variance-to-mean 
ratio of the number of calls up on an infinite trunk group when that 
process is offered to the infinite trunk group. To determine the peaked- 
ness of the CAP, consider the situation shown in Fig. 4. In this figure, 
each time a call is carried on the finite group, a call is put up on the 
infinite group with an exponentially distributed holding time with the 
same mean as on the N-trunk group but independent of the N-trunk 
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group holding time.* As before, j(¢) 1s the number of busy servers on 
the N-trunk group at time ¢. Let k(¢) be the number of busy servers 
on the infinite trunk group at time ¢. We use the following familiar 
result (given for renewal input in Reference 3 and for semi-Markov 
input in Reference 5, which is the form applicable to our problem). 
The following limits 





P; = lim P{k(r7) = 7} (5) 
and : a 
P; = lim Pi k(t) = +} (6) 
exist and satisfy : 
P=, ) 


where @ is the mean time between transitions of the SMP (i.e., mean 
time between carried calls). From (7) we obtain 


lim E{h®(t)} = : Clim M(rr) +1] = : lim M(rt), (8) 


t— 0 


where we have defined 


— M(t) = E{k(p)}. (9) 
ennin 
: M+ = lim M(r+), (10) 


the peakedness of the CAP (denoted 2,) is given by 


2, = Mt — 5 (11) 


Note that 1/8 corresponds to the mean of the carried load (recall that 
we are assuming unity mean holding time). 

We are thus left with the problem of determining M+. This deter- 
mination will be in terms of the distribution of time between carried 
calls, to which the next section is devoted. 


V. DISTRIBUTION OF TIME BETWEEN CARRIED CALLS 


Consider an arrival at 7, which finds a free circuit [1.e., j(7,) < N ]. 
Let F(t) be the distribution of time until the next carry (carried call) ; 


*It is this independence which distinguishes the peakedness from the variance-to- 
mean ratio of busy trunks on the N-trunk group (as discussed in Section I). 

t This is given in Reference 6 for the case of renewal input and weaker assumptions 
on service times. 
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1.€., 
F(t) = P{ict S t|carry at rn}, (12) 


where zcét denotes inter-carry time. Denoting 


F[t| j(tt) < N] = Plict St] j(rt) < N} 





and 
F.(t) = P{ict S$ t|j(rt) = N} (13) 
and recognizing that 
F.[t| j(tt) < NJ] = FO (14) 
yields 
Fw) = FO|1- 3 |+Fo| | as 
where 


P; = P{j trunks busy on the finite group just before 
a call arrival} (16) 


is the stationary call congestion probability given on p. 179 in Chapter 
4 of Reference 3. In particular, Py is the blocking probability. 
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z = PEAKEDNESS OF INPUT PROCESS 
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Fig. 5—Peakedness of CAP (5 trunks). 
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Let ¢.(s) and ¢,(s) be the Laplace-Stieltjes transforms of F(t) and 
F(t), respectively. Transforming (15) gives 








ss) = 66) 1-7 B-| +60 [( AR] aD 


The function ¢.(s) can be obtained from the solution to the Type I 
counter problem given on p. 207 of Reference 3 with renewal input 
transform ¢(s): 


Be(s) = [1 — o(8)] fo mH @)dmy) (18) 
where, for our problem, 
H(t) =1—e™ (19) 
and 
m(t) = E{number of arrivals in (0, ¢)}. (20) 


RCCCECEe 
Ne 
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Fig. 6.—Peakedness of CAP (10 trunks). 
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0 0.4 0.8 1.2 1.6 2.0 2.4 2.8 a2 3.6 4.0 
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Fig. 7—Peakedness of CAP (20 trunks). 


From (18) we have 


$e(s) = 1 - 69] ; 2. - eo | (21) 


where we have used 


i “ e-"dm(t) = are (22) 


Combining (17) and (21) gives the transform of the intercarry time 
distribution : 


(8) = (8) — 





[1 — $(s) ]¢(s + N) Py-1 | 
1 — ¢(s + N) 1 — Py 


We are now in position to determine M+, defined by (10), and sub- 
sequently the peakedness of the CAP. 


(23) 
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0 0.4 0.8 1.2 1.6 2.0 2.4 2.8 3.2 3.6 4.0 
RHO, OFFERED LOAD PER TRUNK (ERLANGS / TRUNK) 


Fig. 8—Peakedness of CAP (50 trunks). 


VI. DETERMINATION OF M+ 


In order to determine the peakedness of the CAP [eq. (11) ], we 
need to evaluate M+ defined in (10). This will be done by characterizing 
the conditional mean of trunks up on the infinite trunk group, given j 
calls up on the finite trunk group. 

Recall that we are considering an arrival at 7, that finds a free 
circuit on the finite trunk group (1e., j(77) < N). The state dis- 
tribution on the finite trunk group at 77 is thus given by 


Py1 


(ope? = 


PF = Prij(rt) = J} = 
P+ = 0, 


IA 
IA 


ISN, (24) 


where the P,’s are the call congestion probabilities defined in (16). 
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0.8 


RHO, OFFERED LOAD PER TRUNK (ERLANGS / TRUNK) 
Fig. 9—Peakedness of CAP (100 trunks). 


We define 
Pty = Pr{j(rt) < MN}, (25) 
M- = E{k(rz)} = E{k(rf)} -1= M+ —-1, (26) 
My = E{k(rt)| g(r) = Uf, (27) 
and 
Mty = E{k(rt)|i(7t) < N}, (28) 


where k corresponds to the infinite trunk group and 7 corresponds to 
the finite trunk group. In terms of these quantities, we have 


M+ = MtyPty + MKP3. | (29) 
Recall that, if 7(7+) < N, the distribution of time until the next 
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z = PEAKEDNESS OF INPUT PROCESS 
1 0.2 0.3 0.4 0.5 0 


0 0. 6 0.7 0.8 0.9 1.0 


0.4 





BLOCKING 


Fig. 10—Peakedness of CAP versus blocking (5 trunks). 


carry is F(Z) and if j(77) = N the distribution of time until the 
next carry is f’,(¢). Using this together with some conditioning argu- 
ments, the following relationship is obtained (see Appendix A): 


M~ = M+ —1 = PiyMtyo(1) + PEHM4¢.(1). (30) 
From (29) and (30) we obtain 


-_ 1 +p+ o-(1) ~~ p(1) : 
Me ay + Mart | a | - 
Use of (21) simplifies (31) to 
+ pt 
ura 1 _ Meio +0). ie 


1 — ¢(1) 1 — (WN + 1) 


It should be noted that the first term in (32) corresponds to the 
value M+ would assume if the renewal input process was offered 
directly to the infinite trunk group. The second term corresponds to 
the reduction in M* as a result of blocking on the finite trunk group. 
We are now left with the problem of determining M¥. 

It is shown in Appendix B that Mt for m = 1, 2, ---, N satisfies 
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Fig. 1l—Peakedness of CAP versus blocking (10 trunks). 


the following set of equations: 
N 
[Mt —1]Pt = 2 Com, m= 1,2,---,N, (83) 
l=max(i,m—1 
where form—1S 1S N-—-1,1SmEN, 
l—m-+1 — 
Cm= PHC) & (OTT) (pe tm: 6x 
m— 1 1 =0 n 
Further, for 1 = N we have 
N N-=m+tl /N — I 

PeK(, 1) 2 (ene tm 


7 =0 7 
1-—¢WN +1) 
It should be noted that the above set of equations is in a form which 
is amenable to solution for the desired quantity M%. Written in 
matrix-vector form, the matrix in question is triangular with additional 
entries below the diagonal. Transforming the matrix associated with 


Cum = (35) 
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BLOCKING 
Fig. 12—Peakedness of CAP versus blocking (20 trunks). 





(33) into triangular form* leads directly to the quantity of interest 
My for use in (32), which is subsequently used to determine the 
peakedness of the CAP [eq. (11) ]. 


VII. EXAMPLES 


We ran some examples with a 2-moment match! interrupted Poisson 
process (Reference 7) as the renewal input to the finite trunk group 
(the computational aspects are discussed in Appendix C). Figures 5 
to 9 show @,, the peakedness of the CAP, as a function of p, the offered 
load per trunk for N = 5, 10, 20, 50, and 100 trunks, respectively. 


* Details are in Reference 9. 
* The blocking experienced by an overflow process is less than the blocking seen 
by a 2-moment match interrupted Poisson process and more than that seen by the 
3d-moment match process (all with the same mean and peakedness). 
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BLOCKING 
Fig. 13—Peakedness of CAP versus blocking (50 trunks). 


On each figure are plots for offered 2’s of 1, 2, 8, and 4. It is seen that, 
for fixed N, as 2 increases, the smoothing effect (reduction of peaked- 
ness) becomes appreciable at lower p’s. This is because of blocking 
remaining negligible for larger p’s as z is decreased. If we fix z, we see 
that the smoothing effect becomes appreciable at lower p’s as N is 
decreased. This is again explainable from the point of view of blocking, 
i.e., blocking is larger on the less efficient small trunk groups. 

Since blocking is an important parameter, Figs. 10 to 14 show the 
peakedness of the CAP versus the blocking for the same cases as 
shown in Figs. 5 to 9. Note that, for final trunk groups which are 
normally operated with blockings of 0.01, the smoothing effect 1s very 
small, while for high-usage trunk groups which may reach blockings 
of a few tenths, the smoothing is substantial. Also, note that z,, in all 
the cases, approaches unity as the load (and blocking) increases which 


1634 THE BELL SYSTEM TECHNICAL JOURNAL, NOVEMBER 1973 


ARC 


RANE 
WSS 


SS 
eS 


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 
BLOCKING 


Fig. 14—Peakedness of CAP versus blocking (100 trunks). 
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is consistent with the explanation in Section I as to how peakedness 
differs from the variance-to-mean ratio of busy servers (which ap- 
proaches zero). 

It was observed in Reference 10 that, when z > 1, the blocking 
probability is bounded away from zero no matter how small the input 
mean is. This is evident in Fig. 10. 


VIIT. CONCLUSION 


We have shown how to determine the peakedness of a CAP. The 
use of mean and peakedness to characterize a CAP is attractive from 
the point of view of simplicity and is consistent with the use of the 
equivalent random method (Reference 2) in trunking analyses. To 
approximately calculate the delays at a switching machine, we could 
replace the CAP (or, more usually, a superposition of CAP’s) with an 
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interrupted Poisson process with the same mean and variance and 
proceed as in Reference 1. This is being investigated. 

We mention, in passing, that we also tried out a renewal approxi- 
mation for z,.. That is, although the CAP is a semi-Markov process 
and not generally a renewal process, we tried to approximate 2, with 


eae ee 
1 — ¢-(1) B’ 


which is the peakedness that the CAP would have if it were a renewal 
process. This approximation did not compare well enough with the 
true z,. to recommend its use. That is, although one might use a re- 
newal approximation to a superposition of CAP’s after the peakedness 
is determined (see the last paragraph), we do not recommend using a 
renewal assumption to determine the peakedness. 

In the course of determining the CAP peakedness, we have more 
fully characterized the CAP as a semi-Markov process. Any queuing 
results available for semi-Markov inputs could be used with the CAP 
semi-Markov characterization given in Section III. 


IX. ACKNOWLEDGMENT 


The excellent programming of Mary Zeitler is gratefully 
acknowledged. 


APPENDIX A 


Derivation of Equation (80) 


Consider the system in equilibrium with times of carried calls {rn}. 
From (26) we have 
M- = EX{k(r)}; (36) 


which can be expanded as 


M- = E{k(ry)|3(r#a) = NIP(i (7h) = N} 
T+E{K (re) | Ftd) < NYP(H(ru1) < N}. (87) 


This can be written as 


M- = Phy 3) ¥ aP{h(re) = t]he) = m, f(r) = NV} 
XP {k(ria) = m| j(r1) = NY 
+Pky © aP{k(re) = 6] bra) =m, jr) < NI] 


t=] 


I 


3 
I 
i=) 


XP{k(ri1) =n| j(ti-1) <N} . (38) 
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Writing 
P{k(re) = t|b(rg-y) = 1, (rg) = N} 
[O Pikln) = ilkGiha) =m, inky) = N 
tn — tra = HAP) 
fe Wy e(L — e-)-igP (2) (39) 


I 


and 
P{k(r,) = t|k(7i) = 0, (7a) < N} 


7 ih & ) e“(L — e)”"d'(t) (40) 


and observing that 


> a (") e~*( 1 — e-')a-t = ne! (41) 
i=l 
leads to the desired result 
M- = PtyMéyo(1) + P4M¥¢.(1). (42) 


APPENDIX B 
Derivation of Equations (88) through (86) for Mx 
Consider the events 
= {j(77_,) = 1, (lL — m + 1) s.c.’s before next 
arrival after rn-1} (48) 
Bim = {j(r#_1) = N, nos.c. before (k — 1)st arrival after 7n-1, 
(N — m-+ 1) s8.c.’s before kth arrival after 7,1}, (44) 


where s.c. denotes service completion on the finite trunk group. From 
these definitions we obtain 


N—-1 00 
P{j(7t) =m} = oe Pi Aim} + 2 P{ Bin}. (45) 
The conditional mean of interest is thus given by 

Mz = E{k(rz)|i(rt) = m} 


2S E{k(rz)|Aim}P {Aim} + + 3 Efk(rz)|Bim}P{Brm} 
- ) 


k * 
PUGH) = ” 


=1 
m 
Defining the events 

= {j(rii) = ]} (47) 


ee = 1, Et A = 0, because P{ j(rt+_,) = 0} = 0. In that case, sum from 
= iN 
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and 
Aeoim = {(l — m+ 1) s.c.,’s before next arrival after rns} (48) 
gives 


E{k(7) [Aim}P {Aim} 


= 5 iP{k (rz) = t, Aatm| Ar} Pt (49) 


=] 


= Pi © Pik(rts) = r|Ad) Y aP{k(rm) = 4 


° 
= 


Asim|A1, k(ri_1) = r}. (50) 
Letting 1 < N and using 


Pik(re) = % Asm Ay k(n) =r} = f° (‘) i ee 


4 ¢ a a e—(m—1)t (1 a e—t)l—mt1q F(t) (51) 


and 
3 (3) ent(1 — et = re (52) 
i=l 
yields 
00 l 
E{k(tz)|Aim}P{Aim} = PPM [ 2 = 1) 


Xe 1 — emt J-mt1d F(t). (53) 
From (1) we have 


E{k(r7)|Aim}P{Ain} = PMP ? :) 


a (' lias : AeGaay 6S 
7=0 7 
ca CinM;, (55) 


where we are using (54) to define C1, 1n (55). This yields (34). 
It now remains to show the desired relationship for Cym. We con- 
sider the second summation of (46) 


E{k(rz)|Bin}P {Bim} = iP {k(z) = 4, By Bren}, (56) 


v 


where 
By = {9 (rit) — N} 
Boum = {no s.c. before (k — 1)st arrival after rr-1, (V — m + 1) 
s.c.’s before kth arrival after rn-1}. (57) 
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Now 


E{k(ty)|Bim}P{ Bim} 


= LD tP{k(rz) = 1, Boem| Ba, k(ta~-1) = 0} 


n=0 i= 


So 
ro" 


XP{k(77_1) = n|Bi}P# (58) 


P&E YX nP{k(rt4) = n| Bi} i ee + - ? jot 


n=0 a—1 


— p—t\(n-t)p—-Ns —(m—1) (t—s) 
xX (1 — e~*) € ( i ) € 


X (1 — ef) N-mHd Fi (s)\dF(t — s). (59) 
Upon performing the summation over 7, (59) simplifies to 


EK (ty) |Bim}P{ Bim} 


= PtMt N a - e—(N+1) sep—m(t—s) 
N N m— 1 é ‘ 


N-—m-+1, = 
(* . + " (—1)e-"*-9)dF,_1(s)dF(t — s) 
n=0 
N N—-—m-+1 N—m + 1 
= PiM4 ae 
PiMi(,,~ 1) 2, r(*~™**) 


Xo (N + 1)o(n + m). (60) 


Summing over k gives 
E Blk(rz)| Bam} P (Ban) 


N—m+1 N= 1 
PEC) ee (MF) oem 
SG) 
1 — oN + 1) 
which is the desired result for (35). 


= Mt 


APPENDIX C 
Computational Considerations 


In this appendix we briefly* discuss some computational problems 
experienced in the numerical solution of the carried process problem 
and point out some possible approaches to circumvent them.' In 


*A more detailed description of our computational experience is in Reference 9. 
T We first discuss the approaches and then the numerical experience we have had. 
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particular, we are concerned with the computation of the state prob- 
abilities, P;, 7 = 0, ---, N, defined in (16) and the computation of 
the quantities Ci, defined in (34) and (385). 

We also specialize and subsequently simplify the above results for 
the case where the interarrival time distribution is a sum of ex- 
ponentials. In particular, we consider the interrupted Poisson process’ 
as the renewal input. This specialization appears to have eliminated the 
numerical problems associated with the required calculations for 
large trunk groups. 

The problem of determining the call congestion state probabilities 
for a renewal input to a BCC system with N independent exponential 
servers is considered on p. 179 of Reference 3. The results are as 
follows: Let 


Cm = (BSED) &. 7 20) 2st N a1 (62) 


1— ¢(j9 + 1) 
with Cy = 1. Then B,, the rth binomial moment of the P,;’s, is given by 
N ({N\ 1 
% ( C; 
B, = Ce ee . (63) 
j=0\ 9 | Cj 
The B, satisfies the backward recursion 
_f1-—er4+)) .) 
Be= (Sey) Bt (Be eA 
with 
1 1 NV {(N\ 1 
— == > 65 
By Py PaGOT. oe) 
(Reference 11, p. 98). The P,’s are given by 
N [7 
Py= X (-Dri( 7) By. (66) 
r=) 


The computation of the C; coefficients and the binomial moments 
B, is fairly straightforward and does not pose much of a numerical 
problem. It is the computation of the P,’s, using (66), that is sensitive 
to numerical errors. The alternating sign in (66), together with the facts 


that " B, can be quite large and the summation in (66) is between 


zero and unity, lead to a numerical problem.* 


* Actually, for an N = 10 case seven significant decimal digits were lost in one 
subtraction and the resultant probability was computed to be zero. This plays havoc 
with the solution to (33). Details are in Reference 9. 
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An alternate approach* to the computation of the state prob- 
abilities is by way of the equations 


N 
Pr= DL pubs, (67) 
j= k—1 
where the transition probabilities p;, represents the probability of 
going from state 7 prior to one arrival to state k prior to the next 

arrival. The p;,’s are given by 


pa= (140) [ema —eperary, F< N, (68) 
0 


and 
PN,k = PN-1,k- (69) 


From (67) and (68), we obtain the backward relation 


Py 1 X 
P | —_— i ns, ° P oe 
= 60) ote) AP? ie 
Expanding (68) gives 


jti-k / 7 = 
pe= (TE VET) peat y- 
1 =0 U 

The computational procedure is outlined as follows: Compute Py 
from (65)? and use (70) to compute P; for 7 < N. It should be noted 
that, although the terms in (71) alternate in sign, ¢ never exceeds 
unity and is monotonically decreasing. Also note the relation between 
pj. given by (71) and Cin given by (34) and (35). At this point, the 
accuracy in computing the P,’s should be comparable to the accuracy 
in computing the Ci,’s. 

For the case where F'(¢) is the sum of exponentials (e.g., interrupted 
Poisson process) we can further simplify (and more accurately com- 
pute) the P,’s and Cim. We go to the integrals from which the sums 
(with alternating signs) appearing in (34), (35), and (71) were derived. 
Note that we have 


Cnt) dy () 6 + 


m—l n=0 
= f . ? i a o-m™(] — e-em F(t) (72) 


* Motivation for investigating this approach stems from remarks made by P. J. 
Burke. (In recent unpublished work, Burke showed a more accurate approach for 
the case where the renewal interarrival time distribution is a sum of exponentials.) 

T Note that each term in the sum of (65) is positive. 
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[see (53) and (54) |. Let 
s 
PW) = 2 Bil — evr); (73) 


then the integral in (72) can be identified as a beta function.* Re- 
peated integration by parts in (72) gives 


ie ( l ) emi Pert) meld h(t). = ¥ Sams ri), (74) 





m—1 

where 
kr; l l — | 
fin (ki, ti) = Gan Mie n= — i) 
l— (l— my) _ 
For 1 = m — I, we obtain from (72) 
kr; 
Fim—1,m(Ki, ti) = aera (76) 
Note that f:, can be computed recursively from 
[+1 
firm(la) = (pets) Sum(besrd (77) 


with initialization from (76). 

The direct calculation of the integral has thus led to a computa- 
tionally tractable method of computing the Cin’s and the P,’s. Cin is 
calculated from 


Cim = Pit ( ¥ fim (he r)) ; m— 1 < l s N — | ) (78)t 
[see (53) ] and 
Pt S 
Crm = aren ( 2, Sum r)) (79)* 


[see (85) and (72) ]. 

The P,’s are calculated from (70) where (68) is simplified as above 
(using integration by parts) and then used to compute the transition 
probabilities. Note that for an interrupted Poisson process,’ S = 2, 
and ky, ke, r1, and re are given in Reference 7 in terms of the switch 


* Identification made by D. L. Jagerman. | 
tT Note that this procedure does not involve the calculation of either binomial 
moments or binomial coefficients. 
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parameters. An alternate method of computing the P,’s for the 
interrupted Poisson process is via the use of birth and death equations 
and conditioning the results on the switch being closed. A computer 
program for doing this was available (Reference 8). 

At this point, it is of interest to discuss our computational experi- 
ence using some of the aforementioned procedures. The first method 
considered was to calculate the P,’s by first obtaining the binomial 
moments [eqs. (62) to (66) | and to compute the C1,,.’s from (34) and 
(35). Using single precision arithmetic the procedure worked for N = 
but failed for N = 10. The problem was traced to inaccurate calcula- 
tion of the probabilities from binomial moments. Double precision 
arithmetic extended the range of N (worked for N = 10). The method 
failed at N = 20. The failure was traced to the same cause as above. 
At this point we used the birth and death equation approach to calcu- 
late the P,’s,8 which assumes an interrupted Poisson input. This 
extended the range to N = 20. For N = 30 we ran into problems 
computing the Cim’s from (84) and (85).* Modification of the Cin 
computation using (78) and (79) significantly extended the useful 
range on NV. The results presented in Section VII were computed using 
this method of calculation. 
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Model Approximations to Visual Spatio- 
Temporal Sine-Wave Threshold Data 


By Z. L. BUDRIKIS 
(Manuscript received May 8, 1973) 


Experimental data on visual spatio-temporal sine-wave thresholds ob- 
tained by Robson and Kelly are considered. In seeking model approxi- 
mations to the data it 1s assumed that the subject’s visual threshold to 
modulation at different spatial and temporal frequencies gives the image 
of his filter function to within a multiplicative constant. [t 1s further 
assumed that the data can be approximated by a system with a spatially 
uniform, isotropic, and temporally invariant response which consists of 
the difference between an excitatory and an inhibitory term, and that each 
term is separable into a product of a spatial and a temporal function. 


I. INTRODUCTION 


Tests of vision with sine-wave flicker go back at least fifty years to 
H. E. Ives.! He determined flicker fusion frequencies with a number 
of wave shapes, including sinusoids. Spatial sinusoid test stimuli are 
more recent. The first to use them was probably Schade? in the fifties. 
Soon after that Kelly*® suggested a stimulus which would simultaneously 
test the spatial and the temporal sine-wave response of vision. Such 
tests were implemented by Robson,’ Kelly,®® and others. 

The special interest in the sine wave as a test stimulus stems from 
the ease with which one can extrapolate from its results. Provided a 
system is linear and time-invariant, Fourier analysis can be used to 
predict the system response to any input from its response to sinusoidal 
inputs. However, the visual system is neither linear nor time-invariant. 
Nevertheless, given a sufficiently constant adaptation state and input 
variations that result in small output variations,’ linear theory can be 
used. 


‘It is often incorrectly stated or implied that, for linearity, the input needs to be 
small. But consider the situation where a flickering light appears fused visually. The 
input may then swing between zero and many times the average luminance, yet the 
behavior is linear. 
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Our interest in the visual system is related to visual communications. 
When visual messages are transmitted digitally then there are po- 
tentially very many different ways—some more advantageous than 
others—in which the messages might be coded and still give acceptable 
fidelity at the receiver. Clearly, it would be good if the likely subjective 
effect of given quantizing procedures could already be predicted at 
the computer simulation stage without involving repeated subjective 
tests. Such predictions will probably be possible soon.? However, there 
is still need for complete specifications both of the linear behavior of 
the visual system and of the nonlinear effects of background masking. 
We will concern ourselves here only with the linear characteristics. To 
this end we will examine several alternative mathematical models to 
see whether they could be used to represent published experimental 
data on spatio-temporal sine-wave thresholds. 

The data that we will use were reported by Kelly® and Robson.‘ 
In both cases threshold values of m were determined in a target 
described by 


L = L,(1 + mcos 2ru,x-cos 2rf,t) , (1) 


where L, is the average luminance, wu, the spatial frequency, and f, the 
temporal frequency. 

Kelly’s measurements! were made at four different values of L,. 
The entire target area, a circular 7-degree CRT face, filled with the 
flickering grating, was viewed monocularly through a 2.3-mm artificial 
pupil. Robson made all measurements at a single L, value. The target 
had a 2.5-degree X 2.5-degree grating in the center of a 10-degree X 10- 
degree screen which had a luminance equal to L,, and it was viewed 
binocularly without artificial pupils. 

In both cases the subject’s threshold was measured by the method 
of adjustment. The subject judged whether he could see the signal or 
not. He did not attempt to distinguish between seeing flicker and 
seeing the bar pattern. During each session of Kelly’s experiment, the 
subject made 5 settings at each of 12 frequencies, with the 60 presenta- 
tions given to him at random. Robson made his measurements in 
orderly sequences. Their results are shown as log-log plots of (1/m) 
against frequency in Figs. 1-5. 

Kelly’s measurements obtained for vision with an artificial pupil 
are converted to equivalent luminances viewed through a natural 
pupil. To calculate the equivalent luminance one needs to take into 
account changes in the size of the natural pupil and the Stiles-Craws- 
ford effect. From data tabulated by LeGrand it can be inferred that, 


TD. H. Kelly kindly supplied a listing of his measurements and standard deviations. 
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Fig. 1—Kelly’s data at 62.8 mL. (a) Temporal frequency response. (b) Spatial 
frequency response. 


given an illuminance J, in trolands, the corresponding luminance L 
in mb is 
L = 1.142 X 107? ['-8; 10 < I < 2000 td. (2) 
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Fig. 2—Kelly’s data at 15.2 mL. (a) Temporal frequency response. (b) Spatial 
frequency response. 
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Fig. 3—Kelly’s data at 3.7 mL. (a) Temporal frequency response. (b) Spatial 
frequency response. 


We will consider six different, though similar, mathematical models 
as possible candidates for representing the data of Figs. 1-5. There is 
a similarity between the models in that: (¢) they all consist of an 
algebraic difference between an excitatory and inhibitory term, (72) 
these terms are in all models separable functions of spatial and tem- 
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Fig. 4—Kelly’s data at 0.91 mL. (a) Temporal frequency response. (b) Spatial 
frequency response. 
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Fig. 5—Robson’s data at 6.3 mL. (a) Temporal frequency response. (b) Spatial 
frequency response. 


poral frequencies, and (272) in each model there will be six undeter- 
mined parameters. We find values for the parameters by digitally 
searching for the smallest weighted mean-square deviation of experi- 
mental points from the models. The fit of none of the models is com- 
pellingly good, but in several cases the degree of fit 1s useful. We find 
the best all-round fit with a model with diffusion-like temporal response 
of excitation, a Gaussian function for the temporal response of in- 
hibition, and Cauchy functions for the spatial response. From the 
point of view of economy in computer simulation, a model with simple 
exponential time responses and Gaussian spatial responses would be 
preferable. However, the mean-square departure from the model is 
somewhat larger than the best. 


Il. THE MODELS 
2.1 The Framework 


By the nature of things, the retinal image is a somewhat blurred 
version of the light distribution in object space. Over isoplanatic 
patches,® or areas A which are large compared to the size of a blurred 
point and small compared to inhomogeneities of the image-forming 
properties of the eye, we can model the formation of the image by a 
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convolution integral with a fixed point-spread function: 


Iau) = [f We - sy — DLE, nde dn, (3) 


where W is the point spread function and L and J are the object and 
image distributions. 

There is virtually no time lag in forming the retinal image. If L 
were switched on at some instant of time then the retinal image 
I(x, y) would be formed at that instant. 

The object-space to image-plane spatial frequency response of the 
given isoplanatic patch, or its modulation transfer function, is the 
Fourier transform of W: 


H(u,v) = [/ W (x, ye 2ri(uztody dy , (4) 


where wu and v are the spatial frequencies in the x and the y directions. 
Point spread in the space domain becomes filtering when transformed 
into the frequency domain. The point spread function W is necessarily 
positive and, with a normal pupillary aperture, has a maximum at the 
center and decreases monotonically. Consequently H (u, v) is a low- 
pass function. 

It is natural to think of perception being based on an “image” at 
some deeper location beyond the retina. This “image” 1s physiologically 
mediated and must suffer appreciable time lags. Hence, the response 
at the deeper location will be time-dependent. There will also be 
further spatial filtering as a result of lateral physiological interactions."4 

Say we designate the resulting point response function by F(z, y, ¢) 
and the internal “‘image’’ distribution by C(a, y, t). At least for a re- 
stricted class of object-space luminance functions, L(z, y, t), C can be 
obtained by superposition, so that 


Out =f ff R@-&y-1t- LG a nardnds. 6) 


The three-dimensional Fourier transform of FR is the spatio-temporal 
frequency response function 


Sie = | f | Ra, y, e2riuetevt ide dy db. (6) 
The integration is over all xz, y, and ¢. f is the temporal frequency. 


We may assume that the response function F# is even in « and y, 
1e., R(x, y, 1) = R(—z, y, t) = R(x, —y, t) = R(—«x, —y, t). This 
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means that S is even in wu and 2, i.e, S(u,v, f) = S(—w, v, f) = 
S(—u, —v, f). No symmetry can be assumed for F in ¢, and hence, 
for S in f. Indeed, A(z, y, ) = 0 for ¢ S 0, and hence, S(u,v, —f) ¥ 
S(u, v, f). 
If the input to the system is the L of eq. (1), then the internal 
‘Image’ is 
C(x, y, t) = S(O, 0, 0)L, +|S(uo, 0, f.)| Lom cos (27ru.2) 
Xcos (2rfot + 6), (7) 


| S(w., 0, fo) | = {S(uo, 0, fo)-S* (uo, 0, fo) }* 


where 


and 
= tan“{Im[S (tho, Q, fo) J/ReLS (u,, 0, ja} } 


The * designates the complex conjugate and Im and Re the imaginary 
and real parts. 

Now we ask: What size must m be before the flickering grating is 
seen with a given level of certainty? We assume thresholds correspond 
to fixed differences, i.e., the flickering grating is seen with probability 
p if 

[S(u, 0, f)|Lom = T(p) , (8) 
where T' 1s a monotonically increasing function of p, but is independent 
of all other variables. We may assume that subjects adjusted m so 
that 1t always resulted in the same probability of seeing. Therefore, 
the values of 1/m, as plotted in Figs. 1-5, are regarded as experi- 
mental determinations of |S(u,0, f)| [to within the multiplier 
T(p)/L. which is a constant when the criterion 7 and the average 
luminance L, are fixed ]. 

If the visual system were truly linear it would have the same re- 
sponse functions irrespective of luminance level L,. But all evidence, 
including that contained in Figs. 1-4, shows that the system adapts. 
It does so somewhat ponderously, much faster with rising L, than 
in reverse, but still quite effectively, changing gain, spatial spread, 
and temporal lag. There is just one aspect of S which Kelly” found 
unchanging over more than four decades of luminance, L,. In large- 
area flicker threshold determinations, using an artificial pupil, he 
found that at different L, values plots of (1/mL.) approached a 
common asymptote for large values of f. However, in other parts of the 
functional domain, different S functions hold for different adaptation 
luminances.® 

In searching for suitable mathematical expressions for f# or S it 
would be convenient if these functions were isotropic, and even more 


1650 THE BELL SYSTEM TECHNICAL JOURNAL, NOVEMBER 1973 


so, if they were also separable into spatial and temporal factors. 
Isotropism would mean that the space variables « and y would reduce 
to a single distance p and the frequencies u and v to a direction- 
independent spatial frequency v. Then 


S(v, f) = S(u = v,0, f) = S(O, 0 = », f) 


i . | / ie )2mpI o(2npr dp |e Pri a (9) 
0 0 


where J, is the Bessel function of order zero. 

Man’s vision is not isotropic. It 1s astigmatic, having better resolu- 
tion in the horizontal and vertical directions than at other angles. 
But, to a first order of approximation, we may assume isotropism. 

Separability of R would mean that we could write it as 


R(p, t) = U(p)V@) (10) 
and then S would also be separable: 
Sv, f) = G)A(f), (11) 
where 
G() = [° U(p)2npJ o(2xpr)dp (12) 
and 
H(f) = [ " V (ten 2S edt. (13) 


Moreover, because U(p) is symmetrical, and hence G(v) is a real- 
valued function, it would follow that 


IS, Nl = E)|H()I. (14) 


However, even a superficial look at the families of experimental 
curves in Figs. 1-5 will convince one that |S(v, f)| is not separable. 
If it were, then curves of |S(v, f)|, as functions of f at different values 
of vy, would differ from each other only by constant multipliers. Plotted 
against a logarithmic ordinate this would result in fixed vertical shifts. 
The same result would hold for plots of |S(», f)| versus » at different 
values of f. But neither of these outcomes are found to be true. This is 
particularly evident when looking at Figs. 5a and b. The curves at 
high values of » or f are low-pass in shape, while for low values of the 
parameters they are bandpass. Figure 6 shows a linearly scaled per- 
spective view of a surface!’ to which the measured values of Fig. 5 
approximate. Measurements apply only to positive frequencies, while 
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Fig. 6—Perspective view of spatio-temporal frequency response. 


the surface has been drawn over all four quadrants making use of 
symmetry. It suggests a volcano with a deep central crater. 

It is customary’ to think of the response as being brought about by 
an interplay of excitation and inhibition, with inhibition responsible for 
the crater. Looked at in this way the measurements suggest, at least to 
a first approximation, that excitatory and inhibitory responses in 
themselves may be separable and that the effects of inhibition simply 
subtract from the effects of excitation. These assumptions will be 
made. The response functions can then be formally broken down: 


R(p, t) = R.(p, t) = Ri(p, t) 


= U.(p) V(t) — Uile) Vid) (15) 
S(y, f) = S.(y, f) = S;(v, f) 
= G.(vy)A.(f) — Gilv) Ai(f) (16) 


with 
G.(v) = J U.(p)2rpJ o(2rpv)dp, 
0 


Het h= i, ” V,(t)e-2*Stdt, 


and similarly for the inhibitory functions. 
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2.2 Choice of Functions 


To satisfy physical considerations, all the component functions 
should be low-pass in character. Of the immense number of possibilities 
we consider just several. The Gaussian function comes readily to mind 
particularly for spatial spreads. 

If 


U.(p) = 





a ae (17) 


then of course the Fourier transform is also Gaussian: 
Ge(y) = e727?0*v? (18) 


The function has another property which can be especially useful in 
computations, namely that as a function of two variables x and y, Le., 
p? = x? + y?, it 1s separable: 








U(x, y) = ( (19) 


1 —72 /2 ‘)( 1 oy, ; 
e727 /2a eny {2a ; 
V27a? V27a? 
None of the other functions of interest to us has this property. 
Another possible candidate for point spreads is the exponential 


U.(p) =e*rh; op > 0 (20) 
and then 
b 
On (b? + yp) 


At high values of v, » >}, the function decreases as (1/v)? which 
corresponds to a fall-off of 18 dB/octave. 

On the other hand, if the spatial frequency response function were an 
exponential then there would be no straight-line asymptote on a log-log 
plot, but rather a response which would be 


G.(v) = (21) 


U.(p) = (22) 


Cc 
2n(c? + p*)? 
with 

Gov) = e-?relvl, (23) 


This is often called the Cauchy response. 

Since the temporal frequency responses are similar to the spatial 
frequency responses similar functions can be used to model these. The 
important differences are that the function V (é) is one-sided and that 
eq. (13), instead of (12), is used to obtain the Fourier transform. 
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The Gaussian function can be used in an approximate way by shift- 
ing it a distance ¢, to the right along ¢ and deleting it leftward of ¢ = 0: 





V(t) = z e~Grrehen;, t20 
TT 
oa) t<0. (24) 


When t./r is greater than three, say, then there is negligible error in 
assuming that V,(¢) is the Gaussian function for all t, ¢ < 0 included. 
Then 

Hf) = eter e-aarstto, (25) 


In computer simulation a simple exponential time response, often 
known as the Poissonian, would be the easiest because it can be effected 
by recursion. That function and its transform are 


Vet) = /rnje!’ 3; t20 (26) 
HD = TR (277) 


A function in which there is theoretical interest“ is one that occurs 
in diffusion processes. Kelly” found that the high-frequency asymptote 
for large-area flicker responses could be fitted well with a frequency 
function which one would find in diffusion that had no losses in the 
diffusing substance, namely with 


H.(f) = Cre lens rl?) (28) 
If the Laplace transform is taken as 
H,(s) = Cye~ 87) (29) 
then the time function is 124 
wie T/2t 
= _W—_— * > 
VO = oy 120. (30) 


The six models which were compared with the experimental data are: 
(7) Gaussian temporal/Cauchy spatial (G/C) 

[S(v, f)| = Ae Perr ice te — ke 2s? te-v7s) | (31) 
(tz) Poissonian temporal/Cauchy spatial (P/C) 


_ Af{[env%e(1 -+ 4a? f273) — kev PP + (2a froke—’%)?}? 
SO DLT aa + epee 
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(21) Diffusion-Gaussian temporal/Cauchy spatial (D-G/C) 
[S(v, f)| = Ale cUm*enre — e-2a*s?rie-vae), (33) 


and three further models in which the Gaussian is substituted for the 
Cauchy response giving G/G, P/G, and D-G/G. 

Note that in four of the models, G/C, P/C, G/G, and P/G, one 
time-lag stage is common to excitation and inhibition (Fig. 7a). The 
remaining two models, involving diffusion, have distinct paths for the 
two effects (Fig. 7b). 

Each of the models differs from the others in its exact functional 
shapes but they are all similar in their form. Figure 8 illustrates the 
evolution of the point spread as given by the P/G model. The point 
spread function is shown at the instant of occurrence of the point 
impulse and at two subsequent time instants thereafter. In this, as in 
all the other models, the excitatory effect is confined to a smaller 
region and has a faster time course than the inhibitory effect. 


III, SELECTION OF PARAMETERS 


Each of the models chosen for comparison with the experimental 
data has six undetermined parameters: the gain A, time constants 71 
and re, space constants o, and o;, and the per unit inhibition k. The 









L (x,y, t) 


(b) 


g. 7—System block diagram: (a) for P/G, G/G, P/C, and G/C models; (b) for 
D- "Sd and D-G/C models. 
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parameters have to be given values to produce as close a fit as possible 
between model and data. 

The following performance index may be used as an appropriate 
measure for the closeness of fit: 


PS a {Lm — 1/|S(v, fi) | Ie}? (34) 


where m, 1S the measured threshold modulation at the spatial frequency 
vy; and temporal frequency f; and e¢; is the estimated (standard) error 
of that measurement. The summation is over all N points measured at 
a given luminance L,. 

This will be called the aggregate-square fractional error, or ASFE, 
index. The ASFE index is perhaps the most defendable in light of the 
experimental procedure. However, if the aim is to obtain the best 
representation of data plotted as (1/m) along a logarithmic scale 
(Figs. 1-5), then a better index is 


P = ¥ flog [|S(vs, f2)|/me] (35) 


which can be called the aggregate-square log error, or ASLE, index. 

Irrespective of index, the array of six parameter values can be looked 
upon as a vector T and the performance index as a real-valued function 
of it. Our object is then to find that location T,, in six-space at which 
P(T) assumes its smallest value. However, there is no way of recogniz- 
ing a global minimum and it is therefore impractical to insist on finding 
it. The object is rather to find as good a value for T as possible, while 
keeping computer expenditures within reasonable bounds. 

Of the many possible parameter search routines we tried a gradient- 
dependent algorithm, random search, and a combination of the two. 
Random search proved the more successful, almost as good on its own 
as in combination with gradient techniques. 

The gradient in question is 

VP = > i a 
n=] OT, _ 
where a, is the unit vector along the nth coordinate axis and 7’, is 


the scalar (T-a,). The components of the gradient were evaluated in 
one of two ways: 


(36) 


(2) approximate differentiation: 


dP . P(T+AT,a,) — P(T) 


aT, = AT, } (37) 
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Fig. 8 (continued). 


(22) evaluation of exact expressions which, given (34), are 


Sr =H ALms = 1/80, 1) Ves} 
x Lie Svs, 49/2] SERED (ag) 


Using (37), care had to be taken in choosing the size of AT'y. 
Given the gradient at the vector location T;, the next location with 
a lower value of P should be at 


T 541 = T; = KVP |; (39) 


This will prove to be so, provided K is small enough. Improved con- 
vergence rates are possible by making K variable, increasing its value 
with repeated improvements in P, and decreasing it with failures. The 
next location to be tested is then not given by (38) but by 


Ty = T, — K;VP|t=1,, (40) 


where T, is the location at which the last lowest value of P was calcu- 
lated and K; has been determined from a starting value K, by multipli- 
cation with either a(0 < a < 1) or y(1 < y), depending on outcomes 
of the 7 iterations thus far. 
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A difficulty with gradient-dependent search is that it may end at a 
local minimum which is far above the global, and that often proved to 
beso. A way around this is to alternate between the gradient-dependent 
search mode and random search. In random search the next location 
to be tested would be 


6 
Tyo = T; + K zy GR; (nan, (41) 


where T;, is again the last best location, K 1s a constant that scales the 
size of the search volume, G,’s are further scaling factors designed to 
make the search about equally sensitive along the different coordinates, 
and #;(n) is a Gaussian variate obtained from a (pseudo)-random 
number routine taking a fresh value for each component and each 
iteration. 

Typically, a computational cycle would consist of gradient-depen- 
dent search to within a convergence test specification, taking some 20 
to 100 iterations, followed by 100 iterations of random search. The 
number of cycles depended on progress and could be as many as 50. 

Most of the performance improvements were found to come from 
the random search phases of the computational cycles. For that reason 
the gradient-dependent phase was dispensed with in many calculations, 
and then K of (40) became a variable similar to K; of eq. (39). The 
calculation was still done in cycles, starting each cycle with a large 
value of K. 


IV. RESULTS 


Although there is no guarantee that the performance indexes finally 
arrived at are the lowest possible, in each case the chances are small 
that there would be anything substantially lower. Hence, Table I can 
be taken as a good guide for comparing the effectiveness of the different 
models in fitting the data. The table gives rms deviations D, which are 
calculated from P in accordance with 


D = [P/(N — 6)}. (42) 


Division is by (N — 6), because the parameters provide six degrees 
of freedom. For Table I, P was as defined by eq. (84), 1.e., the ASFE 
criterion. 

From the last column of Table I it can be seen that the best of the 
six models is the Diffusion-Gaussian/Cauchy and the worst the 
Gaussian/Cauchy. The Poissonian/Gaussian is somewhat worse than 
the average over the group. A comparison of the models by order of 
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TABLE I—RMS DeEviaTIons 
Summary of rms deviations, D, derived from ASFE performance index 
[eq. (34) ]. In computation, experimenter’s estimates of experimental 
errors were used with Kelly’s data and assumed errors with Robson’s 
data. 










Luminance ’ Robson’s 
(mL) Kelly’s Data Data Meéan D 
for 
Model 
62.8 15.2 3.7 0.91 6.3 

Poissonian /Gaussian 7.0 4.3 3.9 7.6 3.7 5.3 
Poissonian /Cauchy 7.2 4.4 4.0 7.9 3.4 5.4 
Gaussian /Gaussian 4.0 4.6 4.7 8.8 2.9 5.0 
Gaussian /Cauchy 6.1 6.2 5.6 9.8 4.2 6.4 
Diff-Gauss/Gaussian 4.0 3.6 4.3 4.4 3.9 4.0 
Diff-Gauss/Cauchy 3.4 3.7 4.5 4.5 2.8 3.8 


rank within each set, and then over the sets, shows the two Diffusion- 
Gaussian models fit best, closely followed by the Poissonian/Gaussian 
model. 

Except with Robson’s data, where assumed error values were used, 
the actual magnitude of D in Table I has significance. With P by eq. 
(34) being measured relative to experimental errors one would expect 
with a perfect model fit a D value of unity. (D — 1) is then the in- 
crease in relative error due to the model, and D can be thought of as — 
error gain. In this sense all the models, including the best, give only 
poor fits. 

The D-G/C model is shown fitted to Robson’s data in Figs. 9a and b. 
According to Table I this ought to be about the best fit, but obviously 
is only fair. The same data is fitted by the P/G model in Figs. 10a and 
b. The P/G model is shown fitted to Kelly’s data at 62.8 mL in Figs. 
lla and b. According to Table I the P/G model represents nearly the 
worst fit. 

Parameter values for the P/G model are given in Table IIA. These 
were determined using the relative error criterion. Table IIB gives 
parameter values for the same model but determined by the log 
departure criterion. The final mean log departures are shown in the 
bottom row. There are noticeable differences between the parameter 
values in Table IIA and Table IIB but, given the rather poor fit 
between model and data, agreement is good. Consistent trends are 
apparent in both sets: with decreasing luminance the gain (A) of the 
system decreases accompanied by a decrease in fractional inhibition 
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|S (f, 0, v) | OR 1/m 
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Fig. 9—Diffusion-Gaussian/Cauchy model applied to Robson’s data. (a) Temporal 


frequency response, parameters as in Fig. 5a. (b) Spatial frequency response, parame- 
ters as in Fig. 5b. 
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Fig. 10—Poissonian/Gaussian model applied to Robson’s data. (a) Temporal 
frequency response, parameters as in Fig. 5a. (b) Spatial frequency response, parame- 


ters as in Fig. 5b. 
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Fig. 11—Poissonian/Gaussian model applied to Kelly’s data at 62.8 mL. (a) 
Temporal frequency response, parameters as in Fig. la. (b) Spatial frequency re- 
sponse, parameters as in Fig. Lb. 


(k). The time constants tend to increase with lower luminance while 
the space constants remain unchanged. The parameter values for the 
D-G/C model obtained using the log of departure criterion are given 
in Table III. With this different model, parameter values are naturally 
very different, but the variations with luminance are similar to those 
with the P/G model and, indeed, with the remaining models. 


TaBLE ITA—PARAMETER VALUES IN POISSONIAN/GAUSSIAN MODEL 
DETERMINED WITH ASFE PERFORMANCE INDEX 





Robson’s 






Kelly’s Data Data 
62.8 15.2 3.7 0.91 6.3 
1 298 236 145 116 219 
2 39 32 32 61 45 
3 63 43 70 102 52 
4 o, (min arc) 1.48 1.55 | 1.49 1.49 1.01 
5 o; (min arc) 9.82 4,72 10.1 6.19 5.62 
6 k 0.9976 0.9831 0.9579 0.8150 0.9554 
7 D 7.0 4,3 3.9 7.6 3.7 
8 A/Lo 4.82 15.5 40.3 127.2 34.8 
9 (1 — k) 0.0024 0.0169 0.0421 0.1850 0.0446 
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TaBLE IJ B—PARAMETER VALUES IN POISSONIAN/GAUSSIAN MODEL 
DETERMINED wiTtH ASLE Crirerion [Ra. (35) ] 






Luminance 







Kelly’s Data ee 


Parameter 


62.8 15.2 3.7 0.91 6.3 
1 A 234 168 134 93 198 
2 71 (ms) 29 37 53 79 55 
3 re (ms) 34 58 80 101 55 
4 o- (min arc) 1.52 1.40 1.37 1.34 1.01 
5 o; (min arc) 9.68 8.30 10.51 10.28 5.58 
6 k 0.990 0.971 0.911 0.740 0.996 
7 D (log units) 0.48 0.56 0.63 0.73 0.49 
8 A/Lo 3.72 11.05 36.2 102 31.4 
9 (1 — k) 0.010 0.029 0.089 0.260 0.004 


V. DISCUSSION 


Both the D-G/C and the P/G models will be useful in practice, 
particularly the latter when simplicity of computation is a major con- 
sideration. However, the fact that none of the models fits the data 
well enough to satisfy any fundamental inquiry prompts us to look 
again at the assumptions of Section 2.1. 

One can scarcely doubt the interplay of excitation and inhibition 
in the visual mechanism, and that inhibition spreads over a wider 


TABLE IITJ—PARAMETER VALUES IN DIFFUSION-GAUSSIAN/CAUCHY 
Mopet. DETERMINED WITH ASLE PERFORMANCE 
INDEX [Ea. (85) | 












j , Robson’s 
a Kelly’s Data Data 
Parameter 
62.8 15.2 3.7 0.91 6.3 
1 A 1596 943 810 372 853 
2 71 (ms) 472 489 649 656 496 
3 t2 (ms) 74 75 74 111 98 
4 o- (min arc) 9.33 8.01 6.47 7.43 8.59 
5 o; (min arc) 12.38 11.45 6.50 8.27 32.4 
6 k 0.517 0.479 0.351 0.236 0.677 
7 D (og units) 0.45 0.48 0.50 0.61 0.33 
8 A/Leo 25.4 62 219 409 135 
9 (1 —k) 0.483 0.521 0.649 0.764 0.323 
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area and persists longer than the excitation, i.e., is confined to lower 
spatial and temporal frequencies. However, it is probably untrue that 
inhibition simply subtracts from the excitation. It is more likely!” that 
it acts as a shunt, or a reduction in through-put gain, for which simple 
subtraction is only a first approximation. One could also expect a more 
precise characterization of inhibitory action to explain part of the 
adaptive changes. However, the model would be nonlinear and more 
complicated. 

Apart from linearity, it is very probable that more separability of 
functions has been assumed than is warranted. The statement that 
excitation (or inhibition) is separable into space and time functions 
purports that, given a point flash, the form of the spatial response is 
independent of time, or that the shape of the time function is inde- 
pendent of distance from the stimulus point. This is probably true of 
the spread which is due to optical smearing of the retinal image. But 
it is probably untrue of the lateral spread of neural interactions. Since 
neural interactions predominate in the wider inhibitory spread, separa- 
bility should be expected to be a poorer assumption for inhibition than 
for excitation. This seems to be borne out by the data. 

The assumption of uniformity raises another question. To speak of 
isoplanatic patches is, of course, no more than a simplification. Even 
the central fovea varies substantially in receptor packing density 
within the space of less than a degree. It is therefore difficult to main- 
tain the assumption of uniformity with data obtained for spatial 
frequencies of one cycle/degree or lower. To justify convolution in the 
presence of nonuniformity we only need to be sure that the spatial 
spread is small compared to the size of the “‘uniform”’ patch. However, 
we need uniformity over much more than (1/f,) in order to justify a 
Fourier transform to within f, of the frequency origin. If this condition 
is not met, then with a sinusoidal input the output may, in the extreme, 
be nonsinusoidal even over only a part of a cycle. But our assumption 
of threshold is that a criterion value be exceeded by the peak-to-peak 
output and this then will not be related to the calculated transfer 
function. 

The concept of detection needs to be examined, not only where 
lack of retinal uniformity is critical. It is unlikely that detection is 
based on a comparison of just two values, a maximum and a minimum 
in the output, and that this comparison is independent of how far 
apart in space and time these two values actually are. It is more likely 
that there should be a pooling of evidence and that there should be a 
decline in detectability, the further apart the relevant events. 
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However, it need not follow that, given a more complicated detection 
mechanism, the modeling done here would be invalidated. The detector 
with variable weighting of evidence could, in fact, be equivalent to a 
spatial/temporal filter in its own right, followed by the kind of decision 
stage assumed here. If this were so, then it would only mean that not 
all the filtering evident from threshold data can be attributed to 
peripheral processes, but that some of it is due to central neural 
activity. This is an important distinction where comparisons are made 
between the filtering evident in stimulus detection and in, say, percep- 
tion of brightness. Inconsistencies of this nature have already been 
noted in the literature,!* but have not been satisfactorily explained. 

Higher-level filtermg might also be responsible for the frequency- 
selective fatiguing discovered by Blakemore and Campbell.’® It seems 
improbable that spatial filtering by optical and retinal spread con- 
stitutes spatial frequency channels which may be independently 
adapted, but higher-level filtering could, in fact, occur after a Fourier- 
like signal transformation. But again, the presence of any transforma- 
tions like these would not affect the present modeling. They might 
however, affect adaptation effects. 
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A/Lo OR (1-k) IN ARBITRARY UNITS 





0.5 1 2 5 10 20 50 100 
ADAPTATION LUMINANCE IN mL 
Fig. 12—Adaptation of gain parameters A/L. (line I) and (1 — &) (line II) 


against luminance as obtained in fitting Poissonian/Gaussian model to Kelly’s and 
Robson’s data. 
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Of the adaptive changes which are evident from the present model- 
ing, the variation in gain requires comment. The fact that the gain 
constant A was seen to decrease with decreasing adaptive luminance 
might be taken to mean that the system becomes less sensitive in the 
dark. As is well known, visual sensitivity goes up markedly with 
darkness and the present results do not, in fact, contradict this. By 
eq. (8), (1/m) equals [S(u, 0, f)| only to within the multiplicative 
constant T'(p)/L,. Assuming that the threshold does not change, then 
to make the gain values at different luminances L, comparable to each 
other they have to be divided by L,. A/L. does in fact go up with 
decreasing luminance as can be seen in row 8 of Table ITA and else- 
where. The actual A/L, values are different across the models but the 
trend is always the same. 

As the adaptation luminance decreases there is an additional increase 
in sensitivity restricted to low frequencies. This occurs because of 
the decline in fractional inhibition &. The zero-point value of |S| is 
with all models A(1 — k)/L.. The net excitation (1 — &) is given in 
row 9 of Tables ITA, ITB, and III. A/Z, and (1 — k) have also been 
plotted for the P/G model in Fig. 12. From the plot one can infer that 
for the P/G model and in the range 1.0 S$ LZ, S 100 mL 


A/L, = const; X L7°*!, (43) 
(1 — k) = const, X L7}*, (44) 

so that 
|S(0, 0, 0)| = const; X L714. (45) 


The increase in low-frequency sensitivity with decreasing luminance 
is at the expense of bandwidth. 


VI. CONCLUSION 


Six spatio-temporal models of human visual filtering were tested 
against published experimental data on visual spatio-temporal sine- 
wave thresholds. These models arose as specific examples from a 
definite theoretical framework. It was assumed that thresholds could 
be related to a fixed peak-to-peak difference in a visually filtered version 
of the input stimulus, and that the filtering could be taken as time- 
invariant and spatially uniform and isotropic. Particular attention 
was directed to the question of whether the response was separable 
into functions of time and space. We showed that the total response is 
not so separable in this way. However, it was assumed that if the 
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response is expressed as an algebraic difference of two terms, excitation 
and inhibition, the individual terms would be separable. 

Component functions which were tried were exponential, Gaussian, 
and diffusion-like functions of time, and Gaussian and Cauchy func- 
tions of space. The best fit was obtained with a model which has a 
diffusion-like time function for excitation, a Gaussian time function 
for inhibition, and Cauchy space functions for both. The diffusion 
function, as a model of the time course of excitation, has previously 
been advocated by Ives,“ Kelly, and others. The degree of fit 
obtained in the present study, involving both time and space, was 
however only moderate and no strong argument can be brought 
forward in favor of any of the functions, not even the best-fitting. In 
the best case the average departure from the model was three times 
larger than the average estimated experimental error. The present 
results do not exclude any of the functions either, for the fit was 
probably affected more by the restrictions of the framework than the 
choice of function. 

In each of the models six parameter values had to be determined. 
These were gain, fractional inhibition, two time constants, and two 
space constants. Parameter searches consisted of up to 50 passes of 
gradicnt-dependent convergence and evolutionary random search. 
Random search was invariably found to be the more productive phase 
in all the computational passes. 

With adaptation luminance between | and 60 mL, the time constants 
were found to be slightly larger at the low luminances than at the high, 
the space constants were almost nonvarying, and the gain and frac- 
tional inhibition decreased with decreasing luminance. As expected, 
the sensitivity, measured as gain divided by luminance, was found to 
go up with decreasing luminance. The reduction in fractional inhibition 
was shown to give a further increase in sensitivity with decreasing 
luminance, but only at low frequencies. With one model (P/G) the 
sensitivity at zero frequency was found to vary inversely as the 1.84 
power of luminance, 0.81 of this being due to variation in overall 
sensitivity and the remainder due to changes in inhibition. 

The major purpose of the present model fitting was to find a filter 
function for use in a program for predicting the subjective quality of 
visual signal coding schemes. Of the six models the most economical 
computational procedures are provided by the Poissonian/Gaussian 
model. The Poissonian, or negative exponential, time functions can be 
implemented recursively, using a delay of only one or two picture 
frames, and the Gaussian space functions, being themselves separable 
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into products of functions of x and y, can be implemented by two 
successive, modest transverse filter operations, instead of requiring one 
very large operation. This model was found to fit the data nearly as 
well as the best. Considering its computational advantages, it will no 
doubt be the one to find most use. 
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The Accuracy of the Equivalent Random 
Method With Renewal Inputs* 


J. M. HOLTZMAN 


(Manuscript received July 19, 1973) 


I. INTRODUCTION 


The equivalent random method?! (aso see Ref. 2) is widely used to 
approximate the blocking probabilities for non-Poisson traffic streams. 
Although much numerical experience and some analysis (e.g., Ref. 3) 
suggests that the method is usually reliable for superpositions of over- 
flows, the reason for its accuracy (or errors) deserves further attention. 

The equivalent random method first determines the mean M and 
variance V of the number of the trunks that would be occupied if the 
traffic were offered to an infinite trunk group. Then an overflow process 
with the same M and YV is offered to the finite trunk group and its 
blocking calculated.' This blocking is taken as the approximation for 
the blocking seen by the original traffic. 

In this Brief, we derive the range of the blocking probabilities which 
may be experienced by renewal streams characterized by the same M 
and V. Since this range may be rather wide, it follows that the success 
of equivalent random method cannot be explained solely by the con- 
straints put on blockings by fixing M and V. Rather, one should factor 
in the special structure of the processes. Furthermore, it is seen that 
one cannot use an arbitrary renewal process to represent another pro- 
cess with the same mean and variance. 


“A version of this Brief was presented at the Seventh International Teletraffic 
Congress, Stockholm, June 1973. 
' That is, the blocking is calculated for the specific renewal process which is the 


overflow process from a Poisson input. Conceivably, other types of renewal processes 
could be used. 
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II. IMPLICATIONS OF THE EQUIVALENT RANDOM METHOD 


Consider a nonlattice renewal process, with distribution function 
F(t) for the interarrival times, offered to a group of N trunks. The 
holding times are mutually independent exponential random variables 
with unity mean (or the mean is the time unit). Blocked calls are cleared 
and the system is in equilibrium. Define 


f  tdF (2), (1) 


m 


b() = f° emar(o, (2) 


Then it is known that the blocking probability is 


r= fe(3) 5g 


+(N) B= 61 Osan 
N $(1)¢(2):--¢(W) 

(see, e.g., Ref. 4, Chap. 4). Observe that B depends on WN values of 

¢(z),7 = 1, ---, N, and that it is an increasing function of these ¢(2). 

We shall show how the equivalent random method constrains these 

¢({t) by obtaining upper and lower bounds on them which, in turn, give 

upper and lower bounds on B. 

The description of the equivalent random method in the Introduc- 
tion leads to the question of how well M and V characterize a traffic. 
It turns out that they imply much more than is apparent at first glance. 
For our renewal input, we have the following relationships: 


M =m, (4) 


1—¢(1) — 
(see, e.g., Ref. 4, Chap. 3*). Thus, (MV, V) uniquely determines 
(m, ¢(1)) and vice versa. Specifically, | 


V/M—-1i1+M 
0) = “Trt Mt 6) 


v= ™| i | (5) 


Hence, the equivalent random method fixes ¢(1) which is particularly 
important in (3). Moreover, fixing V and M puts important constraints 
on the other ¢(z), 7 = 2, ---, N, which, in turn, further constrains B. 


* Also, see Ref. 5, p. 8331/5, for an interesting characterization of peakedness. 
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Fig. 1—Constraints on ¢(z) for x > 1. 


Figure 1 shows how (m, ¢(1)) constrains ¢(z) for x > 1. The parame- 
ters p1, Po, 6 will be given in (16)—(18). All such ¢() must lie within the 
shaded area. The upper bound is a least upper bound and the lower 
bound, a greatest lower bound. Also shown in Fig. 1 is a wedge for 
x > 1 which respresents simpler, cruder bounds for ¢(x) which follow 
immediately from the decreasing convex nature of ¢(2). 

To derive the lower bound, let y = e~S in 


E\y| SH \y|*, «e>41, (7) 
with é the renewal interarrival time. 
We obtain 
Kt(e+) & E(e-*) (8) 
or, in other words, 
L¢(1)]* S ¢(2), (9) 


so that ¢(z) must lie above the indicated curve for x > 1 in Fig. 1. 
To show that this is a sharp lower bound, let 


dF (t) = [pid (t — a) + pod (t — b) Jat. (10) 
(pi, @, D2, 6) must satisfy 


Pit pe = 1, (11) 
pia + prob = m, (12) 
pie? + pre? = $(1). (13) 
By letting b get large and p, — 1, we can show that 
o(x) = pie~** + pre-*? — pie** — [ (1) ]*. (14) 


The sharp lower bound for ¢(x) may also be derived using Theorem 
2.1 on p. 472 of Ref. 6 (see Remark 2.3, p. 474). Use of this theorem* 


*The problem to which we applied this theorem is to find sharp upper and lower 
bounds for fo°e~*‘dF (t) subject to fo°dF () =1, fo*tdF (ft) =m, and fo°e ‘dF (t) =¢(1), 
a number fixed by (6). 
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also leads to the following sharp upper bound for ¢(z): 


b(%) S bm(t) = pi + pre”? (15) 


with (1, De, b) satisfying 
m(1 — e-°) 


= Te” - 
m 
Poi b? (17) 
pi = 1 — pro. (18) 
We thus obtain that the true B satisfies 
Br,SBS Bi, (19) 
where 
_ N\1-— ¢(1) 
By = 1+ (7 )+5 eo = a 
N\ [1 — (1)]--- [1 — 6% (1)]) 72 
(Hopf, on 
Be={1+(7) ao) 
N\ [1 — on()]---L1 — on) ]|7. 
+ (yy) eae on 


The blocking probability obtained by the equivalent random method, 
Ber, also satisfies (19) so that a bound on the error for the method is 


max {B, — Be, Ber — Br}. 


lil. INTERPRETATION OF EXTREMAL SOLUTIONS 


Some feeling for these bounds can be obtained by considering the 
maximum and minimum blocking probabilities attainable when only 
the mean interarrival time m is constrained (the equivalent random V 
is unspecified). It is shown in Ref. 7 that the minimum blocking is 
achieved when arrivals are regular with a separation of m. Our inf may 
be viewed as approaching that of regular arrivals but with a different 
mean [the impulse at b in (10) keeps the equivalent random M and V 
satisfied ]. Observe that Bz is the blocking probability seen by a renewal 
input with constant interarrival times with mean m, determined from 


emt = (1). (22) 
It is shown in Ref. 7 that with a given m, blocking probabilities 


ACCURACY OF EQUIVALENT RANDOM METHOD 1677 





20 30 40 5060 80 100 
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Fig. 2—B,, Ber, Bu for V/M = 1. 


arbitrarily close to unity may be obtained by having F(t) consist of a 
step at ¢ sufficiently small and another small step at ¢ sufficiently large. 
This causes most of the arrivals to come tripping on each other’s 
heels. Our maximum blocking may be viewed as trying to approach this 
but constrained by V to keep the second step at a finite ¢. 


IV. EXAMPLES AND DISCUSSION 


Some bounds are shown in Figs. 2 through 4. These results do not 
necessarily imply that the equivalent random method is commonly 
subject to errors of such magnitude. In practice, the method is usually 
applied to superpositions of overflows and these are a special class of 
processes, generally not renewal.* Nevertheless, the relatively large 
differences between the inf and sup blockings suggest that the ap- 
parent success of the equivalent random method for superpositions of 
overflows cannot be explained solely by the constraints put on block- 
ings by fixing M and V. Rather, explanation of this accuracy should 
factor in the special structure of such processes. (It may be of interest 
to extend the results of this Brief to take special structures into ac- 
count.) Furthermore, it is seen that one cannot use an arbitrary re- 


* Teletraffic interest need not be confined to simple superposition of overflows from 
trunk groups; e.g., switching center congestion can alter a traffic. 
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Fig. 3—B,, Be:, Br for V/M = 2. 


newal process to represent another process with the same mean and 
variance. 

As an aside, observe that if V/M > 1, the blocking B is bounded 
away from zero no matter how small M is. That is, (6) implies that 





1 2 3 4 5 6 8 10 20 30 40 5060 80100 200 300 
M 


Fig. 4—B,, Ber. Br.for V/M = 4. 
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By, of (20) for fixed V/M cannot get below Bz evaluated with 
g(1) = 1 — (MV). 
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